I'm not a rabid proponent of firewalling or security that hurts productivity or ease of use. In fact, I'd prefer to run my servers with no firewalling at all. The trouble however is that once you establish a server with any real mass merit, someone is going to try to mess around with it, probably with the intent of crashing it, deleting your data, or just distrupting service. It will happen, and it's probably best to be prepared ahead of time, rather than effect overprotective policies once you've had an attack take place. I also want to note that firewalling is not a security silver bullet. Firewalling merely reduces the likelihood of an attack (how can they take advantage of that 0-day SSHd bug if SSHd isn't avaliable to them?, etc).
My firewalling does a few important things. First, I only firewall incoming traffic. Secondly, I firewall based on network services, and their privilege level. This gives you a great deal of flexibility while still leaving your server pretty secure in terms of public access. You can also modify the methodology pretty easily to allow for even more layers of trust access. For me though, a trusted and untrusted level satisfy my needs.
I recommend the following ruleset as a skeleton. You can just copy paste.
Just as a refresher, IPTables works by following down the chains until it finds a final target such as ACCEPT, DROP, REJECT, etc. So a ping packet, for instance, would navigate through the tpit chain, the local traffic chain, the state chain, and then stop at the icmp chain. It would never test against the std or auth chains because the icmp chain allowed it. If the target for icmp was DROP instead, it would still stop there, never hitting std or auth. There are much better primers on how IPTables works. I suggest checking Google for that type of information.
The first thing I do when preparing to lock down a server is to create a network map for it. I have mine setup in a spreadsheet (example located here). Pay attention to the first tab for now, the one called "Servers & Services". What I've done here is listed the servers I have (names, services, and IPs changed for anoninimity.) and information about the network services that need public access FROM the Internet.
So take a few minutes and inventory your publicly listening services. Even things that aren't necessarily daemons, such as a BitTorrent client, or a port for IRC-DCC transfers- those need documenting because in order for them to work, external clients need to reach them over the Internet. Ignore the automatically generated IPTables line for now.
As you're creating your map, consider some things. Does your HTTP server REALLY need to be listening on all IPs bound to the machine? Is FTP really something that you need to allow public access to?
My thoughts on the matter are to not run any services on INET_ANY (i.e., on all interfaces). Specifying IPs in your service configurations allows you several levels of flexibility-- it means you can use port 80 for another service on a different IP (such as a BitTorrent Tracker). If you really are too lazy to edit your configuration files, then just specifying the IP you want public in your Network Map spreadsheet will cover that too (by blocking the others that are listening).
Another thing is that you need to decide what counts as a privileged service. I'd tell you off the bat that anything you wouldn't tell people you have running (but need to run) qualify here. Things like your SSH daemon, SNMPd, maybe even FTPd. In my configuration, I allow public requests to udp53 (dns), but only allow priviledged requests to tcp53 (dns zone-axfr). Easy reasoning-- I want people to be able to query my DNS server, but I don't want to let them grab my zone files.
This lists CIDR masks that you consider to be trusted (or trusted enough) to be able to reach anything you marked as "special" or privileged. I would suggest finding the CIDR for your ISP, your office Internet, other servers you may have, etc. Maybe you have one friend who you want to let use the FTPd on the server. Find her CIDR and add it. Keep your server names consistent across both tabs. If you want to allow a CIDR to access all servers, use "All". If you want it to access only two or three servers, separate with &.
So you're using the Google Spreadsheet method for your ruleset and you think they're pretty good to go. Well great! I wrote a script that you can put in crontab. It periodically checks the Google Spreadsheet for firewall ruleset changes and updates as necessary. Say you're at a relative's house (oops, no CIDR added!). Navigate to your spreadsheet page, add the CIDR, republish and wait a few minutes-- you're good to go.
You can find the source for my script here. Change the spreadsheet links to yours. Also, you'll need to know a few things if you go this route. Each tab must end with the word "END" in the last row. The script checks for this to ensure no data was lost. Another thing: In cell G1, there is a number. Each time you want an update to occur, you need to change this number (needs to be between 1 and 60000). It does not matter if you increment or decrement it. The script uses this number to determine if it should update the ruleset. And lastly, you MUST republish the Google Spreadsheet when you're done. Publishing is the same as saving a public copy of the document. Even you change a bunch of stuff, the script wont see any changes until you republish.
As far as running the script goes, it takes 1 mandatory argument and 1 optional one. The first argument must be the name of the server. The second argument is 'debug', which turns off real updates and only shows you what it would have done instead. I HIGHLY RECOMMEND running it in debug before running it for real.
Once everything is set, issue a policy DROP command. This will mean anything that does not match the rules you set up will not be permitted.
Feel free to contact me with any questions you have. This page was sort of hacked together during some spare time I had, so I'll admit it's probably murkey in some spots. I'd also like to note I have this system running successfully on 3 high-traffic servers and it's been working without fail. I'm a big fan and proponent ;).