Login and download all files script

This is very handy if you're too busy and don't have time to download whatever files you need.

The -D specifies the domains allowed, this is because I specified -H which means foreign hosts are allowed, if you don't restrict them you'll end up going to the whole internet via ads and other links just like a search Engine would follow.

-l 0 specifies to go deep, to as many levels as possible/as exist.

-e robots=off is important because robots.txt often says you can't view certain files which we don't want (this is intended for search engines).

-A .zip specifies that we only want to download .zip files, you can add more, change it from zip to jpg or remove it all together if you like.

--user and --password are hopefully obvious

wget -l 0 -e robots=off --keep-session-cookies -A .zip -H -D allowedomain.com,otherdomain.com -U "Mozilla/5.0 (X11; Linux i686; rv:12.0) Gecko/20100101 Firefox/12.0" -m --user=user --password=pass http://url.com

 


Tags:

login, download, scriptthis, specifies, domains, specified, hosts, restrict, ll, via, ads, links, levels, robots, txt, engines, zip, jpg, user, password, wget, allowedomain, otherdomain, quot, mozilla, linux, rv, gecko, firefox, http, url,

Latest Articles

  • QEMU / KVM How To Manually Create Basic Virtual Machine VM
  • Linux wlan0 check all wireless clients
  • PHP Issues With Decoding Strange Smart Quotes And Non-Standard ASCII Characters
  • /etc/iproute2/rt_tables default settings file in Linux Centos 6,7 and most other NIX's
  • bind named error solutions named[2169]: error (no valid DS) resolving / error (broken trust chain) resolving / : error (no valid RRSIG) resolving 'com/DS/IN':
  • iptables how to log ALL dropped incoming packets
  • How To Edit Linux Based NM Network Manager Connection Settings Without GUI
  • Linux Disable IPV6 Centos / Debian / Mint Howto
  • Linux use growisofs to burn a larger file on a BD-R / Bluray Disc
  • Linux partprobe/partx cannot access last and 4th partition
  • DRBD Errors Caused By Physical Corruption
  • mdadm: add new device failed for /dev/sdb4 as 3: Invalid argument solution
  • Linux named / bind how to dump, view and clear the cache!
  • Centos 6 / 7 / 8 How To Change Default nameservers in /etc/resolv.conf when using DHCP / dhclient
  • Adobe Acrobat Reader for Linux to use and view XFA Fillable Forms
  • Debian Linux Mint Ubuntu iptables save and restore settings automatically onboot and reboot
  • Why SMART is not smart at all and doesn't properly predict disk errors that cause a kernel panic or crash
  • scp: ambiguous target error and solution
  • VirtualBox How To Add iSCSI Storage using VBoxManage
  • iSCSI on Centos 7 Configuration and Setup Guide for Initiator and Target