On 9/7/05, Jeremy Maitin-Shepard <jbms@cmu.edu> wrote:
Beman Dawes <bdawes@acm.org> writes:
I'm not seeing any obvious way to bulk download. Anyone else have any ideas?
Perhaps wget -r will work?
Nope. You need to authenticate by submitting a form, so wget -r doesn't work. Thankfully there are Perl modules (sorry, Dave) that make this none too difficult. See the attached Perl script "ygspider" which can be used to spider the files from a Yahoo! group given the group name, a Yahoo! username and password. It requires the WWW::Mechanize Perl module to work. You can run "perldoc ygspider" or "ygspider --help" to get detailed help, but the basic usage is: ygspider groupname username password Beware that this script probably violates Yahoo! TOS and I seem to have managed to get my home IP address banned in my numerous test runs, but not before I sucessfully mirrored all of the Boost files. Let me know if I can send these someplace for better hosting. They use about 22 MiB of disk space in total. -- Caleb Epstein caleb dot epstein at gmail dot com