bash script remove non-printable special characters/white space

In bash you test like this for problems that can be caused by non-printable characters since if you don't know how to identify it because it basically breaks your script.

echo "url=$url" should print something like this normally if you have properly formatted input/text:

url=::http://someaddress.com::

But if you have some weird hidden characters (not visible in a text editor of any sort you'll get something like this
::l=::http://someaddress.com

How come it doesn't print as expected?
There are hidden/non-printable characters, usually ASCII code I believe that cause unexpected behavior like above and wreaks havoc on many scripts/programs who didn't realize this since there are non-printable characters.

There is a program in Linux that takes care of it called "tr"
If I read data line by line using cat, I now use "tr" to filter all the non-printable characters (I use it line by line and not the entire file since this would remove things like linebreaks and then you have a text file that becomes a single line).

Here is how you'd fix a bad variable (one with non-printable/special characters):

url=`echo "$url"|tr -dc ‘[:print:]‘


Tags:

bash, printable, characters, spacein, echo, quot, url, formatted, input, text, http, someaddress, visible, ll, doesn, ascii, wreaks, havoc, scripts, programs, didn, linux, tr, filter, linebreaks, variable, dc, lsquo,

Latest Articles

  • Proxmox How To Purge Ceph
  • VMWare ESXi/VSphere Disable Balloon Segfault in Services Solution
  • Apache Linux Debian Ubuntu Container how to manually restart without killing
  • Docker enable UTF8 in Container to stop seeing gibberish ? characters
  • Debian 8 How To Use Apt Update Archive sources.list
  • Debian Live CD Password
  • Forbidden You don't have permission to access this resource. [authz_core:error] [pid 338:tid 338] [client 1.2.3.4:55046] AH01630: client denied by server configuration:
  • The client needs a new connection for this request as the requested host name does not match the Server Name Indication (SNI) in use for this connection.
  • Asterisk RTP/audio not working in either direction in Docker NAT with a NAT client
  • dovecot: imap(root@localhost)<3702>: Error: Mailbox INBOX: mmap(size=352609044) failed with file /var/spool/mail/root/Maildir/dovecot.index.cache: Cannot allocate memory
  • Asterisk cannot find soundfile file.c:824 ast_openstream_full: File for-tech-support does not exist in any format
  • Apache Error solution - mktemp: failed to create directory via template '/var/lock/apache2.XXXXXXXXXX': No such file or directory
  • sysctl settings to reduce buffers and caches in Linux
  • Find /dev/sd block device of ata device - ata6: SATA link up 1.5 Gbps (SStatus 113 SControl 310) ata6.00: qc timeout (cmd 0xec) ata6.00: failed to IDENTIFY (I/O error, err_mask=0x4) ata6.00: revalidation failed (errno=-5)
  • Stuttering Audio on VOIP phones when first answering a call slow and fast audio
  • How to distribute the Microsoft VC Visual Studio Redistributable Files On Your Own
  • Nvidia video resolution and codec encode decode support matrix eg. h264 4k h265 HEVC VP9 Card List from GTX, RTX, Quadro
  • Japan and China ping time observations
  • ffmpeg convert to another format eg h265 to h264
  • Apache stop bots and hackers by using forensic logging.