Login and download all files script

This is very handy if you're too busy and don't have time to download whatever files you need.

The -D specifies the domains allowed, this is because I specified -H which means foreign hosts are allowed, if you don't restrict them you'll end up going to the whole internet via ads and other links just like a search Engine would follow.

-l 0 specifies to go deep, to as many levels as possible/as exist.

-e robots=off is important because robots.txt often says you can't view certain files which we don't want (this is intended for search engines).

-A .zip specifies that we only want to download .zip files, you can add more, change it from zip to jpg or remove it all together if you like.

--user and --password are hopefully obvious

wget -l 0 -e robots=off --keep-session-cookies -A .zip -H -D allowedomain.com,otherdomain.com -U "Mozilla/5.0 (X11; Linux i686; rv:12.0) Gecko/20100101 Firefox/12.0" -m --user=user --password=pass http://url.com

 


Tags:

login, download, scriptthis, specifies, domains, specified, hosts, restrict, ll, via, ads, links, levels, robots, txt, engines, zip, jpg, user, password, wget, allowedomain, otherdomain, quot, mozilla, linux, rv, gecko, firefox, http, url,

Latest Articles

  • Cisco Switches How To Get Of Port Line Status Console Messages
  • Cisco DHCP Snooping Relay Setup Information
  • Cisco Switch Setup Guide Command List
  • Cisco 2960 Switch Reset To Factory Defaults
  • How To Boot Cisco CUCM UCSInstall 8.6, 10, 11 and 12 on KVM/Proxmox
  • VBOX VirtualBox How To Import Raw .img Disk File
  • Windows Server 2012, 2016, 2019 How To Install and Missing Disabled Telnet Client
  • proxmox vm networking breaks when you restart your network on the hostnode
  • Linux ln symlink how to update existing symbolic link
  • Ubuntu 18.04 / Linux Mint 19.1 Cannot Type or Login - solution
  • LUKS Hard Drive Encryption on Linux Mint Ubuntu Debian etc how to mount encrypted hard drive
  • How to use nmap locate other machines/computers/servers on your network using nmap
  • Linux Mint 18.2 Create Config File To Start Application Upon Login
  • Dell Wyse Thin Client BIOS Access Key
  • sudoers file in /etc warning about comments/includes!
  • Centos 7 Reallocate logical volume space to another
  • lvm how to reduce volume size
  • letsencrypt certbot error "Unable to find a virtual host listening on port 80 which is currently needed for Certbot to prove to the CA that you control your domain. Please add a virtual host for port 80."
  • SSH error cannot Forward or Listen "bind: Cannot assign requested address"
  • X11 SSH Linux Forwarding Error