Netcat + Tar for simple file transfers
This past week I spent 3 days watching a 3-thread sftp process transfer over a system backup of 53GB. This backup in particular had a ton of files and directories, and I suspect most of the actual time was spent querying and creating the files rather than the actual transfer itself. Now that everything is transferred over, I can rsync it against another machine, but that's another story.
In searching for a better way than what I did, I found this cool gem:
On the sender:
tar cf - directory_to_copy | netcat other_host 7000
On the receiver:
netcat -l -p 7000 | tar x
How to use proxycheck
proxycheck is a small program design to (surprise) check for proxies. What's not so clear, however, is how to use it. The default examples are for mail servers, which I guess was the thing to do back in 2004 when the last version was released. However, I need to adapt it for use with pypsd. Here's how to check a host for open proxies in an IRC context:
./proxycheck -c "chat::real.irc" -d real.irc.server.here:6667 -aaaa host.to.check
The chat parameter confused me for awhile until I figured out what it was for. proxycheck takes care of scanning ports and connecting to hosts using a number of methods, but once connected, you need some way to verify a real hit. -d tells proxycheck to try connecting to the supplied host:port, and -c chat:<send>:<watch> tell it to try sending some text and scanning for watch. If it matches, you have a positive hit.
In the case of IRC, the first thing that happens when you connect to an IRC server is a series of notices from the IRCD (at least in hybrid), so watching for "real.irc" (e.g., replace this with part of your ircd's hostname) is good enough.
If you want more verbose output (checking the data stream), don't bother with Wireshark. Just add -vvvv to the command.
Enjoy.
