bash script remove non-printable special characters/white space

In bash you test like this for problems that can be caused by non-printable characters since if you don't know how to identify it because it basically breaks your script.

echo "url=$url" should print something like this normally if you have properly formatted input/text:

url=::http://someaddress.com::

But if you have some weird hidden characters (not visible in a text editor of any sort you'll get something like this
::l=::http://someaddress.com

How come it doesn't print as expected?
There are hidden/non-printable characters, usually ASCII code I believe that cause unexpected behavior like above and wreaks havoc on many scripts/programs who didn't realize this since there are non-printable characters.

There is a program in Linux that takes care of it called "tr"
If I read data line by line using cat, I now use "tr" to filter all the non-printable characters (I use it line by line and not the entire file since this would remove things like linebreaks and then you have a text file that becomes a single line).

Here is how you'd fix a bad variable (one with non-printable/special characters):

url=`echo "$url"|tr -dc ‘[:print:]‘


Tags:

bash, printable, characters, spacein, echo, quot, url, formatted, input, text, http, someaddress, visible, ll, doesn, ascii, wreaks, havoc, scripts, programs, didn, linux, tr, filter, linebreaks, variable, dc, lsquo,

Latest Articles

  • VMWare Pro Workstation Nic Disconnected and No IP Using NAT
  • Linux How To Create A RamDisk
  • mdadm force resync when resync=PENDING solution
  • Proxmox Breaks Storage/LVM Backing If Killing QEMU-IMG
  • Proxmox trying to acquire lock... TASK ERROR: can't lock file '/var/lock/qemu-server/lock-102.conf' - got timeout
  • Debian 9 SSH root password authentication failure password not working problem / solution
  • QEMU / KVM How To Manually Create Basic Virtual Machine VM
  • Linux wlan0 check all wireless clients
  • PHP Issues With Decoding Strange Smart Quotes And Non-Standard ASCII Characters
  • /etc/iproute2/rt_tables default settings file in Linux Centos 6,7 and most other NIX's
  • bind named error solutions named[2169]: error (no valid DS) resolving / error (broken trust chain) resolving / : error (no valid RRSIG) resolving 'com/DS/IN':
  • iptables how to log ALL dropped incoming packets
  • How To Edit Linux Based NM Network Manager Connection Settings Without GUI
  • Linux Disable IPV6 Centos / Debian / Mint Howto
  • Linux use growisofs to burn a larger file on a BD-R / Bluray Disc
  • Linux partprobe/partx cannot access last and 4th partition
  • DRBD Errors Caused By Physical Corruption
  • mdadm: add new device failed for /dev/sdb4 as 3: Invalid argument solution
  • Linux named / bind how to dump, view and clear the cache!
  • Centos 6 / 7 / 8 How To Change Default nameservers in /etc/resolv.conf when using DHCP / dhclient