Login and download all files script

This is very handy if you're too busy and don't have time to download whatever files you need.

The -D specifies the domains allowed, this is because I specified -H which means foreign hosts are allowed, if you don't restrict them you'll end up going to the whole internet via ads and other links just like a search Engine would follow.

-l 0 specifies to go deep, to as many levels as possible/as exist.

-e robots=off is important because robots.txt often says you can't view certain files which we don't want (this is intended for search engines).

-A .zip specifies that we only want to download .zip files, you can add more, change it from zip to jpg or remove it all together if you like.

--user and --password are hopefully obvious

wget -l 0 -e robots=off --keep-session-cookies -A .zip -H -D allowedomain.com,otherdomain.com -U "Mozilla/5.0 (X11; Linux i686; rv:12.0) Gecko/20100101 Firefox/12.0" -m --user=user --password=pass http://url.com

 


Tags:

login, download, scriptthis, specifies, domains, specified, hosts, restrict, ll, via, ads, links, levels, robots, txt, engines, zip, jpg, user, password, wget, allowedomain, otherdomain, quot, mozilla, linux, rv, gecko, firefox, http, url,

Latest Articles

  • QEMU-KVM soundhw deprecated how to enable sound in QEMU 4.x series
  • Virtualbox Error Cannot register the hard disk because a hard disk with UUID already exists solution
  • kernel: [549267.368859] mate-terminal[7871]: segfault at 2000000101 ip 00007f5d0a9548f0 sp 00007fff7012c610 error 4 in libgobject-2.0.so.0.4800.2[7f5d0a920000+52000]
  • apcupsd how to setup and monitor APC UPS units
  • How To Password Reset, Recover, Bypass, Remove and Unlock on Windows 10,8,7,Vista,XP,NT,2000,2003,2008,2012,2016,2019 Administrative Login Programs
  • Nvidia Ubuntu Linux Screentearing Video with solution driver
  • ?? Question Marks for time, permissions and size of a file?
  • mdadm how to stop a check
  • access denied by acl file qemu-kvm: bridge helper failed
  • Linux NIC connecting at 100M instead of 1000M gigabit speeds? It could be overheating
  • "This kernel requires the following features not present on the CPU: cmov Unable to boot - please use a kernel appropriate for your CPU.
  • http://vault.centos.org/5.9/os/i386/repodata/filelists.xml.gz: [Errno -1] Metadata file does not match checksum solution
  • Linux Ubuntu Wifi Disabled Only Works When Laptop Plugged Into Wall AC Power
  • CentOS 6 impossible to compile a newer libguestfs
  • chroot
  • How To Get Started on Ubuntu with gpt-2 OpenAI Text Prediction
  • Remove cloud-init in your VM
  • QEMU-KVM KVM Command Line Practical Guide
  • Linux How To Change NIC Name to eth0 instead of enps33 or enp0s25
  • virt-resize: error: libguestfs error: could not create appliance through libvirt.