Message boards : Number crunching : Proxy to reduce downloads
Author | Message |
---|---|
dcdc Send message Joined: 3 Nov 05 Posts: 1832 Credit: 119,821,902 RAC: 15,180 |
Hi all i'm resurrecting an old project with some old redundant hardware. My plan is to have 4-8 minimal PCs in a box with each PSU shared between two PCs. They'll all be running on compactflash and running Damn Small Linux except the 'server' which is running XP Pro (i don't know linux well enough to set up ICS or something similar, and i've got XP down to 75MB RAM). I'll be running these comps in place of an electric heater, so there's no CO² issues! I think this will have to connect to the internet using dial-up so i want to reduce the downloads as much as possible. I think I'll be running 24 or 48hr work units, but need to minimise downloads as much as possible. I put a request in on the BOINC dev boards for a folder that all big downloads are copied to which can then be checked before a download is initiated - that'd save a lot of bandwidth, both for users with lots of hosts and of course on the project's side. Anyway, until that's implemented, the question is: Can I set up a proxy in XP that'll store the big rosetta files to stop repeat downloads? Any suggestions welcomed! cheers Danny |
hugothehermit Send message Joined: 26 Sep 05 Posts: 238 Credit: 314,893 RAC: 0 |
I'm not sure under XP, but you could always use SmoothWall it's a firewall, proxy, DHCP server etc... Just run your computers on dynamic IPs, run SmoothWall as a DHCP server with Web proxy settings as transparent and with the cache XX size set as enabled. I run SmoothWall 2.0, the new one is 3.0 and it might be a little different, but the documentation was good with 2.0 so I would assume that 3.0 would be just a good. One thing to remember is that when you boot from the SmoothWall setup disk it will erase the HDD completely then repartition it to it's heart content. |
dcdc Send message Joined: 3 Nov 05 Posts: 1832 Credit: 119,821,902 RAC: 15,180 |
I'm not sure under XP, but you could always use SmoothWall it's a firewall, proxy, DHCP server etc... ah... looks interesting! As this is gonna have to be on dial-up i guess one of the key factors is whether i can get a modem working with it. I've got a choice of three here but i couldn't find drivers for two of them for XP! I'll investigate - cheers hugo. one question - how much RAM does the version you're running use? i've got 256MB in this machine - will that be enough? |
Feet1st Send message Joined: 30 Dec 05 Posts: 1755 Credit: 4,690,520 RAC: 0 |
dcdc, in addition to the ideas here, I've been working on a cacheing proxy. Still making changes and burning it in, so it's not yet ready for prime time, but it should give you exactly what you are looking for. Shoot me an EMail and we can compare notes. I can't recall if you joined the Yahoo group or not. I'm hoping to work with members of the group to further test my code. But you can use this EMail addr: HelpOutRosetta-owner at NOyahooSPgroAMups.comERS (remove NOSPAMERS from the domain name). Add this signature to your EMail: Running Microsoft's "System Idle Process" will never help cure cancer, AIDS nor Alzheimer's. But running Rosetta@home just might! https://boinc.bakerlab.org/rosetta/ |
dcdc Send message Joined: 3 Nov 05 Posts: 1832 Credit: 119,821,902 RAC: 15,180 |
sent ;) |
hugothehermit Send message Joined: 26 Sep 05 Posts: 238 Credit: 314,893 RAC: 0 |
My SmoothWall machine http://stats.kwsn.net/host.php?proj=all&hostid[]=5356007 P3 500MHz 128MB RAM IDE ~450MB HDD (this is too small, but I had it in the back of draw so to speak) Generic old NIC (was auto detected) hooked to 10/100 4 port hub. 1.44 floppy IDE CD-ROM It's hooked to a 128k ISDN modem through Com1 D9 pin 115200 baud rate. External modems work without drivers just esc strings mine is +++ATZ!T0=45!T1=20!T4=1 what that works out to my modem is +++ esc AT attention modem Z initialise !T0=45 add B channel if more 45kbs (dial second telephone line and add it to the internet) !T1=20 drop B channel if less than 20kbs ( hang up second telephone line) !T4=1 enable DVO (Data Voice Override) (second telephone line automatically hangs up to allow people to ring out and in) I should also add that I use this all the time, once the machine is setup it will give internet access to anything I've put on it 98SE, XP, Vista, Linux. I used to run ICS on windows then found something called SOHO something or was that something SOHO, doesn't matter SmoothWall it's better than sliced bread if you ask me. Edit: because the url tags don't like ...hostid[]=535... in it. Edit: for the modem stuff. Edit: more added Edit: spellnig loots off it :) |
mdettweiler Send message Joined: 15 Oct 06 Posts: 33 Credit: 2,509 RAC: 0 |
I'm not sure under XP, but you could always use SmoothWall it's a firewall, proxy, DHCP server etc... I've checked out Smoothwall before (actually installed it in Virtual PC), and it seems that it's supposed to work very well with dial-up modems. Also, it doesn't need much RAM--I think the minimum is 32MB. |
Klimax Send message Joined: 27 Apr 07 Posts: 44 Credit: 2,800,788 RAC: 184 |
It is interesting how people from different places can have same idea. What I would suggest is to try network boot,so you can dump compactflash in favour of one bigger HDD in server.That is the only part,which was showstopper for me...(Unable to figure out how and no time to do experiments) |
dcdc Send message Joined: 3 Nov 05 Posts: 1832 Credit: 119,821,902 RAC: 15,180 |
It is interesting how people from different places can have same idea. netbooting would be good - BitSpit on my team (XPC) netboots his crunchers - but the hardware is all different so i'd have to configure it for each machine, and if there's any network interruptions it tends to need resetting. Also, i don't think the HD is quick enough to serve 8 machines as it's a 4300RPM drive from an old laptop. I am running out of CF-IDE adapters though so if i've got any lan cards with a boot rom on them (i believe that's pretty much all that's needed?) then I might have a look at getting a couple of them netbooting so there's not too much load on the HD. If i've got two identical motherboards i'll have a look at it... |
Message boards :
Number crunching :
Proxy to reduce downloads
©2024 University of Washington
https://www.bakerlab.org