i’m sure that most (if not all) of the technically inclined readers know about this already, but since it’s such a well written article (and detailed in the right places), i felt i should bump up it’s link count.
http://en.wikipedia.org/wiki/Binary_prefix
as an aside, i really liked Knuth’s suggestion; in the meantime let’s use the stupid kibi’s.
lvs load balancing always felt like an elusive task. here i will document how to get it working with the excellent shorewall firewall, as an extension to their two interface common use case. this was all necessary for a group of grad students that needed to test out and develop some distributed algorithms. it turns out that once you get going, all this is quite easy and fun!
the various components and files used for this setup include:
let’s get going. first setup the head node. for networking, you’ll need a public ip and a private one. in my case /etc/network/interfaces looks like this:
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface (public)
auto eth1
iface eth1 inet static
address 123.321.52.210
netmask 255.255.255.0
network 123.321.52.0
broadcast 123.321.52.255
gateway 123.321.52.253
# dns-* options are implemented by the resolvconf package, if installed
dns-nameservers 123.321.52.1 123.321.52.3
dns-search something.example.com
# The private lan
auto eth0
iface eth0 inet static
address 192.168.1.254
netmask 255.255.255.0
you’ll notice that in the two interface default shorewall config, my eth0 is their eth1 and vice-versa. also, i’ve replaced my hostname and ip block with something invented. sorry! anyways, the tail of dhcpd.conf looks like this:
option subnet-mask 255.255.255.0;
option broadcast-address 192.168.1.255;
option routers 192.168.1.254;
subnet 192.168.1.0 netmask 255.255.255.0 {
#range 192.168.1.10 192.168.1.100;
host node1 {
hardware ethernet 00:12:34:56:78:91;
fixed-address 192.168.1.101;
}
host node2 {
hardware ethernet 00:12:34:56:78:92;
fixed-address 192.168.1.102;
}
}
next to make it easier for myself to reference the nodes i’ll setup /etc/hosts:
123.321.52.210 node.something.example.com node
192.168.1.101 node1
192.168.1.102 node2
shorewall comes next. use the default two-interface setup, and add the following entries in /etc/shorewall/rules:
ACCEPT net $FW tcp 8000
ACCEPT $FW loc tcp 8000
i’ve decided i want to distribute port 8000. to make it easier for ipvs to figure out which packets should get load balanced, we can mark them with /etc/shorewall/tcrules:
1 0.0.0.0/0 132.206.52.210 tcp 8000
which marks them with integer: “1”. lastly for ipvs, edit /etc/sysctl.conf, and add:
net.ipv4.ip_forward = 1
let the kernel know with:
sysctl -p
and then tell ipvs what is going on with:
ipvsadm -A -f 1 -s rr
ipvsadm -a -f 1 -r node1:8000 -m
ipvsadm -a -f 1 -r node2:8000 -m
the various flags are well documented in the ipvsadm man page and are self explanatory here. you can test this all out by running something at the two nodes, i used:
python -m SimpleHTTPServer
which serves the current directory with a text file named “NODE1” and “NODE2” respective to the node, and i tested that the requests alternate by pointing my browser to the head node at port 8000.
i hope this meets everyones needs for documentation and knowledge; i couldn’t have done this without great ipvs reference or the amazing shorewall.
while doing a little reading and research, i ended up reading a bit of the hdparm man page. never in my life have i read such a scary man page. i guess it is appropriate for halloween, but it came 2 days late in my case. as a result of my being so impressed with the sheer amount of warnings in the manual, i have decided to compile them here.
–dco-restore
Reset all drive settings, features, and accessible capacities back to factory defaults and full capabilities. This command will fail if DCO is frozen/locked, or if a -Np maximum size restriction has also been set. This is EXTREMELY DANGEROUS and will very likely cause massive loss of data. DO NOT USE THIS COMMAND.
–drq-hsm-error
VERY DANGEROUS, DON’T EVEN THINK ABOUT USING IT. This flag causes hdparm to issue an IDENTIFY command to the kernel, but incorrectly marked as a “non-data” command. This results in the drive being left with its DataReQust(DRQ) line “stuck” high. This confuses the kernel drivers, and may crash the system immediately with massive data loss. The option exists to help in testing and fortifying the kernel against similar real-world drive malfunctions. VERY DANGEROUS, DO NOT USE!!
–fwdownload
When used, this should be the only flag given. It requires a file path immediately after the flag, indicating where the new drive firmware should be read from. The contents of this file will be sent to the drive using the (S)ATA DOWNLOAD MICROCODE command, using either transfer protocol 7 (entire file at once), or, if the drive supports it, transfer protocol 3 (segmented download). This command is EXTREMELY DANGEROUS and HAS NEVER BEEN PROVEN TO WORK and will probably destroy both the drive and all data on it. DO NOT USE THIS COMMAND.
-m
Get/set sector count for multiple sector I/O on the drive. A setting of 0 disables this feature. Multiple sector mode (aka IDE Block Mode), is a feature of most modern IDE hard drives, permitting the transfer of multiple sectors per I/O interrupt, rather than the usual one sector per interrupt. When this feature is enabled, it typically reduces operating system overhead for disk I/O by 30-50%. On many systems, it also provides increased data throughput of anywhere from 5% to 50%. Some drives, however (most notably the WD Caviar series), seem to run slower with multiple mode enabled. Your mileage may vary. Most drives support the minimum settings of 2, 4, 8, or 16 (sectors). Larger settings may also be possible, depending on the drive. A setting of 16 or 32 seems optimal on many systems. Western Digital recommends lower settings of 4 to 8 on many of their drives, due tiny (32kB) drive buffers and non-optimized buffering algorithms. The -i flag can be used to find the maximum setting supported by an installed drive (look for MaxMultSect in the output). Some drives claim to support multiple mode, but lose data at some settings. Under rare circumstances, such failures can result in massive filesystem corruption.
-R
Register an IDE interface (DANGEROUS). See the -U option for more information.
-u
Get/set interrupt-unmask flag for the drive. A setting of 1 permits the driver to unmask other interrupts during processing of a disk interrupt, which greatly improves Linux´s responsiveness and eliminates “serial port overrun” errors. Use this feature with caution: some drive/controller combinations do not tolerate the increased I/O latencies possible when this feature is enabled, resulting in massive filesystem corruption. In particular, CMD-640B and RZ1000 (E)IDE interfaces can be unreliable (due to a hardware flaw) when this option is used with kernel versions earlier than 2.0.13. Disabling the IDE prefetch feature of these interfaces (usually a BIOS/CMOS setting) provides a safe fix for the problem for use with earlier kernels.
-U
Un-register an IDE interface (DANGEROUS). The companion for the -R option. Intended for use with hardware made specifically for hot-swapping (very rare!). Use with knowledge and extreme caution as this can easily hang or damage your system. The hdparm source distribution includes a ´contrib´ directory with some user-donated scripts for hot-swapping on the UltraBay of a ThinkPad 600E. Use at your own risk.
-w
Perform a device reset (DANGEROUS). Do NOT use this option. It exists for unlikely situations where a reboot might otherwise be required to get a confused drive back into a useable state.
-x
Tristate device for hotswap (DANGEROUS).
–security-unlock PWD
–security-set-pass PWD
–security-disable PWD
–security-erase PWD
–security-erase-enhanced PWD
–user-master USER
–security-mode MODE
[…]
THIS FEATURE IS EXPERIMENTAL AND NOT WELL TESTED. USE AT YOUR OWN RISK.
i think it is now safe to say that:
# rm -rf /
has now officially been replaced with the much more dangerous:
# hdparm -mRuUwx --dco-restore --drq-hsm-error --fwdownload --security-unlock PWD --security-set-pass PWD --security-disable PWD --security-erase PWD --security-erase-enhanced PWD --user-master USER --security-mode MODE [device]
have fun kiddies!
The Technical Blog of James 