/ - rtt - IT Resource

  • systemd management using systemctl and journalctl to check systemd logs


    systemd is like the service manager for your Centos and other modern Linux distributions (including Debian/Mint/Ubuntu) allows you to enable services, stop them, restart them, check their status and even reboot your system. The key commands or arguments you will use with systemctl are the following: Unit Commands:   list-units [PATTERN...]         List loaded units &nbs........
  • VMWare ESXi 6.7 SSH/PowerShell CLI Commands


    [root@localhost:~] BootModuleConfig.sh          echo                         host-ind                     nfcd                   ........
  • Ethereum Client Errors


    INFO [07-19|12:11:51] Imported new block receipts              count=590  elapsed=8.440s     bytes=74117699  number=4870906 hash=6bc60b…934753 ignored=0 INFO [07-19|12:11:57] Imported new state entries               count=499  elapsed=9.963ms    processed=71076 pending=1526  retry=2&nbs........
  • Centos 7 - How To Install NFS and Mount Remotely


    1.) First install nfs-utils [root@bcitcourse ~]# yum -y install nfs-utils Loaded plugins: fastestmirror base                                                         &n........
  • Install NFS (Network File System) On Debian Linux Ubuntu Mint Howto


    sudo apt-get install nfs-kernel-server #oops there are no exports so it won't startsudo /etc/init.d/nfs-kernel-server start * Not starting NFS kernel daemon: no exports. #we will use the /tmp/nfstestshare directory for our NFS share mkdir /tmp/nfstestshare #add it to /etc/exports (basically what NFS checks to determine what to make an NFS share) /tmp/nfstestshare 192.168.1.5(rw,sync,........
  • mdadm how to stop or start a check


    It's fairly simple to start or stop a check but I do wish mdadm's command had this built in.  Sometimes it will do a check at the worst time causing the server to crawl to a halt. Stop check on md126: echo idle > /sys/block/md126/md/sync_action Start check on md126: echo check > /sys/block/md126/md/sync_action  ........
  • l2tp ipsec VPN Error Sep 12 18:16:25 vps pluto[7299]: ERROR: asynchronous network error report on eth0 (sport=500) for message to 192.5.6.2 port 20640, complainant 192.5.6.2: Connection refused [errno 111, origin ICMP type 3 code 3 (not authenticated


    Sep 12 18:16:25 vps pluto[7299]: ERROR: asynchronous network error report on eth0 (sport=500) for message to 192.5.6.2 port 20640, complainant 192.5.6.2: Connection refused [errno 111, origin ICMP type 3 code 3 (not authenticated)] Some say changing the "leftprotoport=17/%any" will fix this but I have not found this to be the case. Essentially it means at least one end is blocking the ipsec packets.  Sometimes the %any allows an alt........
  • rsync specify alternate port non-standard port than 22


    It is not obvious but the rsync --help  rsync --help|grep port      --port=PORT             specify double-colon alternate port number --port does not do anything at all actually for some strange reason it still uses 22 You have to specify a manual ssh command to make it work: -e 'ssh -........
  • Centos 6.6/6.9 KVM VM Kernel Panic On Boot - Kernel panic - not syncing: Attempted to kill init!


    I am not sure why this is happening neither the hostnode or VM changed.  All I did was reboot the hostnode and startup the Centos VM again, also note it happened with the original kernel on the VM and also the latest 6.9 kernel as of this writing as shown below. Host Node: Centos 6.9 Kernel: 2.6.32-696.6.3.el6.x86_64  Kernel: 2.6.32-042stab123.9 Same result in any kernel above........
  • Linux Mint USB Kernel Tainted and Locked Port/Dev File


    Essentially a program I was running for mining did not terminate properly with Ctrl+C it is listed as defunct and cannot be killed, kernel is tainted and normal tricks to disable the port are impossible the dev and sys entries for the device cannot be browsed or interacted with in any form without a lockup of the request.  The only solution is to reboot due to the kernel taint as far as I can find so far.   [1130246.811056] INFO: task minerd:21861 blocked for more th........
  • ffmpeg Linux Mint download, compile and install howto


    #if you have nvidia make sure you install the nvidia-cuda-toolkit so hardware acceleration can be used wget http://ffmpeg.org/releases/ffmpeg-3.3.2.tar.bz2 tar -jxvf ffmpeg-3.3.2.tar.bz2 cd  ffmpeg-3.3.2/ ./configure --disable-yasm install prefix            /usr/local source path              ........
  • Migrate Centos 7 from Single HDD to mdadm RAID 10 array:


    Done on Centos 7.3 very important as clearly based on older guides it was a lot easier and more simpler!  Hint do not use grub2-install! One huge caveat if you are an oldschool user or sysadmin who has avoided UEFI booting The normal way will not work here if your Centos was using UEFI.  Newer systems use it by default. The easiest way to check is to do an fdisk -l if your sy........
  • rsync run as root sudo without password


    This is a common issue, what if a issue shouldn't have root but you want to use that user to make a full backup of a system?  They of course need root access. You can actually just give them passwordless sudo access to rsync in /etc/sudoers: sudo vi /etc/sudoers yourusername ALL = NOPASSWD: /usr/bin/rsync   Here is how you would execute rsync: The key thing for the remote host is to........
  • cudaminer freezes Linux Mint taints kernel


    When running cudaminer once it tries to initialize the card the entire screen freezes.  The computer itself is still running but the Xorg is done for, you cannot even switch to another console window and must reboot (even an mdm or Xorg restart does not help). At first cudaminer will give you these errors: stratrum_recv_line failed ...retry after 15 seconds GPU #0: Geforce 210 with compute ca........
  • Linux Mint Install Netboot PXE Guide Howto


    *Update so this doesn't work it must be something to do with the path of nfs or something else but the installer fails with "Installer crashed" at the end whereas with the CD/USB it works. This assumes you've already installed and configured a separate PXE/DHCP server somewhere else and your /tftpboot directory is setup. This is for Linux Mint 18.1 but generally applies to most versions although you may have tro change things like "casper"........
  • ecryptfs errors


    ecryptfs-mount-private Enter your login passphrase: Inserted auth tok with sig [ee16d84] "into the user session keyring mount: No such file or directory" [  156.118113] ecryptfs_mount: kern_path() failed [  156.118431] Reading sb failed; rc = [-2] [  164.233055] traps: mate-notificati[3472] trap int3 ip:7f43d7002c13 sp:7fff162c6600 error:0 [  166.017061] ecryptfs_mount: kern_path() failed........
  • What happens when you unplug 1 or more devices from an mdadm RAID array to simulate a failure in Linux Ubuntu/Centos/Debian?


    In short the two drives in the array were /dev/sdd and /dev/sde.  The kernel sees they were unplugged and have gone down as you can see below. mdadm caught the first one being unplugged /dev/sde and disabled the missing drive.  However when the final drive that was part of the array is unplugged it didn't notice at all.  Instead it complains about an IO error later for drives that the kernel knows do not exist anymore. [45817.162728] ata4: exception........
  • Migrating from Linux Mint 17.2 mdadm RAID array to a new one (because I upgraded to larger drives).


    1.) Replicate the number of partitions in your new drives. gdisk /dev/sda gdisk /dev/sdb I created 3 partitions of the same same size. partition #1: +1G (/boot) partition #2: +60G (swap) partition #3: rest of it (/) #note if you are using GPT/gdisk you need to create separate a partition at least 1MB in size (in my case I would a 4th partition and mark it type ef02).........
  • mdadm how to recover from failed drive


    Remove the failed partition /dev/sde1  mdadm --manage /dev/md99 -r /dev/sde1 mdadm: hot removed /dev/sde1 from /dev/md99   Now add another drive back to replace it:  # mdadm --manage /dev/md99 -a /dev/sdf1 mdadm: added /dev/sdf1   A "cat /proc/mdstat" should show it resyncing if all is well.........
  • mdadm frozen and doesn't realize array is dead/missing failed due to unplugged drives


    This was a surprising bug but I unplugged all drives for an array md127.  At first it was just 1 drive and mdadm seemed to notice this.  I unplugged the second drive taking the array offline but mdadm did not realize it was offline and still showed a non-existent disk as being part of it.  This created problems trying to unmount it or even to stop this array with mdadm freezing.  As for how to fix it I can only think of making sure you are not in a mounted path of........
  • DRBD Split-brain solution


    Uh oh [17925926.174277] block drbd0: Handshake successful: Agreed network protocol version 96 [17925926.174325] block drbd0: conn( WFConnection -> WFReportParams ) [17925926.174342] block drbd0: Starting asender thread (from drbd0_receiver [1682]) [17925926.174432] block drbd0: data-integrity-alg: <not-used> [17925926.174581] block drbd0: drbd_sync_handshake: [17925926.174586] block drbd0: self 2AAE66AF9252D6DB:2815BF........
  • fdisk wq causes hung process


    I created a new partition table on a newly plugged in device and it caused fdisk to hang (even force kill does not work).  It also may be a bad drive or some other issue because fdisk -l hangs after the first 2 HDDs (totaly of 8 HDDs on this system): [1232879.903596] INFO: task fdisk:27176 blocked for more than 120 seconds. [1232879.903607]       Tainted: P        &nbs........
  • DRBD Slow Performance - 99.99 % [jbd2/drbd0-8] highiowait solution


    Tired of checking iotop and seeing that your drbd partition is using 99.99% of io all the time and finding your drbd device performs slow in general? This is especially an issue in versions of DRBD in the 8.3 tree in particular one documented case is on "8.3.13" but it likely applies to other devices. The symptoms are that resyncing is fine and normal but any reasonable amount of activity is very slow and lagged and creates a high server load and con........
  • Linux How To Recover Partition Table from Kernel and Restore Centos/Debian etc..


    We've all done this at some point, you work on the wrong shell window and this was my first time making this mistake but I deleted a partition table in fdisk, recreated it and saved it with "wq" and even ran partprobe!  If you haven't rebooted yet then you can still recover your partition table, otherwise you're in big trouble. Fortunately since it was a live system and in use the kernel still had to use the old table like below:........
  • OpenVZ Error: Failed to load 2nd level quota solution


    Syncing private Live migrating container... Syncing 2nd level quota 11000: invalid option -- F Usage: vzdqload quotaid [-c file] commands Loads user/group qouta information from stdin into quota file.     -c file    use given quota file Commands specify what user/group information to load:     -G    grace time     -U    disk limits........
  • blocked for more than 120 seconds issues with iowait and timeout on Live Linux distribution while writing to physical disk


    This happened while an mdadm array was syncing, all access from writing a new blank file to opening a small .txt file was very slow:   [222117.312078] kjournald starting.  Commit interval 5 seconds [222117.685060] EXT3-fs (md0): using internal journal [222117.685096] EXT3-fs (md0): mounted filesystem with ordered data mode [222122.376847] kjournald starting.  Commit interval 5 seconds [222122.602825] EXT3-fs (md2): using internal jour........
  • SELinux targeted policy relabel is required


    You may get this after a long time of not rebooting but especially if you have rsync'd a / partition or deployed an image to another VPS or computer, you will often have this issue. The good thing it usually just takes a reboot. Here is more info from Centos........
  • Centos 6 how to guide convert LVM non-RAID into mdadm 1/10 RAID array live without reinstalling


    Here is the scenario you or a client have a remote machine that was installed as a standard/default minimal Centos 6.x machine on a single disk with LVM for whatever reason.  Often many people do not know how to install it to a RAID array so it is common to have this problem and why reinstall if you don't need to?  In some cases on a remote system you can't easily reinstall without physical or KVM access. So in this case you add a second physical or disk or already ha........
  • Centos cannot login by console or ssh session closed immediately


    In my case I could login with the initial install but I rsync'd everything over while preserving ownership and permissions to another RAID partition and booted from that.was fine before. The problem is that you are kicked out the second you login and the problem was SELINUX for some reason (perhaps it noticed something strange when it was moved to the new partition) login: pam_unix(login:session): session opened for user root by LOGIN(uid=0) login: ROOT LOG........
  • rsync backup entire system properly


    rsync -Phaz --exclude=/dev/* --exclude=/sys/* --exclude=/proc/* --exclude=/mnt/md126 / /mnt/md126........
  • Samsung Printer Password Login to change settings CLP


    First of all to find your IP you can hold the "X" button for a few seconds and it will print out the DHCP IP. Another way to find the IP that I prefer is to login to your router or switch and find a hostname that says something like "SEC001599CD2948" which will be your Samsung printer (at least for the C3xx or C4xx series of Samsung printers). The login and password for Samsung Syncthru Webservice:........
  • drbd won't sync 8.3.13 on OpenVZ kernel


    <p>I used the matching 8.3.13 utilities and it didn't work but strangely the newer 8.3.16 which makes DRBD complain works just fine.</p> <p>GIT-hash: 83ca112086600faacab2f157bc5a9324f7bd7f77 build by root@sighted, 2012-10-09 12:47:51<br /> &nbsp;0: cs:SyncSource ro:Secondary/Primary ds:UpToDate/Inconsistent A r-----<br /> &nbsp;&nbsp;&nbsp; ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:5236960<br /> &nbsp;&nbsp;&am........
  • Centos 6 server crash - " kernel:[735188.961824] Kernel panic - not syncing: Fatal exception in interrupt Write failed: Broken pipe"


     kernel:[735188.961824] Kernel panic - not syncing: Fatal exception in interrupt Write failed: Broken pipe I'm not sure what is causing this and have never seen it on any server before.........
  • What a hdd hard drive and mdadm RAID array failure looks like in Linux


    [3805108.257042] sd 0:0:0:0: [sda] 1953525168 512-byte hardware sectors: (1.00 TB/931 GiB) [3805108.257052] sd 0:0:0:0: [sda] Write Protect is off [3805108.257054] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00 [3805108.257066] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA [3805108.257083] sd 0:0:0:0: [sda] 1953525168 512-byte hardware sectors: (1.00 TB/931 GiB) [3805108.257090] sd 0:0:0:0: [sda] Write Protect is off........
  • Linux Centos 6 Weird Kernel Panic


    Kernel panic - not syncing: Attempted to kill init! Pid: 1, comm: init Not tained: 3.2.51-3.el6.x86_64 #1 Call Trace:  panic+0x8c/0x1a2 forget_original_parent+0x34d/0x360 exit_notify+0x12/0x140 do_exit+0x1d8/0x400 do_group_exit_+0x50/0xc0 sys_exit_group+0x12/0x20 system_call_fastpath+0x16/0x1b panic occurred, switching back to text console........
  • sudo: sorry, you must have a tty to run sudo solution


     ./rsync.sh sudo: sorry, you must have a tty to run sudo rsync: connection unexpectedly closed (0 bytes received so far) [sender] rsync error: error in rsync protocol data stream (code 12) at io.c(601) [sender=3.0.7] Solution edit /etc/sudoers #Defaults    requiretty........
  • mdadm how to add additional RAID 1 drives


    This is a great way to upgrade your RAID array or move it/copy it to a new set of hard drives. Eg. you have a current RAID 1 array on older/slower drives. Just add at least 1 of the new drives to the array, update grub/install it and then boot into it.  Then you have a transparent data migration that is fully synchronized.   mdadm --grow /dev/md126 --raid-devices 3       md127 : active raid1 sdc1........
  • mdadm increase recovery sync/resync speed slow solution/fix


    The units in echo are kB as in kilobyte. Setting a high sync speed echo 120000 >/proc/sys/dev/raid/speed_limit_min This will increase the speed, note that sometimes a rebuild is slow due to current disk activity/iowait. If that is not the cause then you may have a hardware issue (controller, cable or a bad drive). Setting a lower sync speed echo 1200 >/proc/sys/dev/raid/speed_limit_max........
  • CPanel suhosin install failed


    /scripts/phpextensionmgr install PHPSuHosin Updating md5sum list Fetching http://httpupdate.cpanel.net/cpanelsync/easy/targz.yaml (connected:0).......(request attempt 1/12)...Resolving httpupdate.cpanel.net...(resolve attempt 1/65)...     Fetching http://httpupdate.cpanel.net/mirror_addr_list (connected:0).......(request attempt 1/3)......connecting to 74.50.120.123...@74.50.120.123......connected......receiving...100%......request success......Done........
  • Android how to copy rsync binary to main system for use in shell/ssh


    Install rsync4randroid and in the shell/ssh do this: ln -s /data/data/eu.kowalczuk.rsync4android/files/rsync /system/xbin/rsync........
  • Dell FS12-NV7 2U Server Information and Guide


    I've got one of these for testing projects from work at home and got more than I bargained for with the time I've spent on it due to the storage handing/Perc 6/i cards. My particular model came with the following: 2U Rack Mount Server with Rails 2xOpteron 2373 EE (Quad Core, there is a 6-core version that can be found at times) 16GB RAM 2 x 250GB Seagate SATA 2 x Dell Perc 6/i (horrible and a nightmare to work........
  • pxe-32 tftp open timeout


    pxe-32 tftp open timeout   The solution was to enable tftp in xinetd with "chkconfig tftp on". See the troubleshooting below:   chkconfig --list NetworkManager     0:off    1:off    2:off    3:off    4:off    5:off    6:off acpid              0:off&n........
  • EXT4-fs error (device md20): ext4_add_entry: bad entry in directory #2552670:


    Jun 12 10:30:53 kernel: [724514.291670] EXT4-fs error (device md20): ext4_add_entry: bad entry in directory #2552670: rec_len % 4 != 0 - offset=216, inode=2553603, rec_len=94, name_len=84 Jun 12 10:30:53 kernel: [724514.292400] EXT4-fs error (device md20): ext4_add_entry: bad entry in directory #2552670: rec_len % 4 != 0 - offset=216, inode=2553603, rec_len=94, name_len=84 This happened when an unprivileged user was backing up a Windows filesystem, once they ha........
  • mdadm enable bitmap to speed up rebuilds


    Have you ever unplugged the wrong drive and then had to rebuild the entire array?  It may not be a big deal in some ways but it does make your system vulnerable until the rebuild is done. Many distros often enable the "bitmap" feature and this basically keeps track of what parts need to be resynced in the case of a temporary removal of a drive from the array, this way it only needs to sync what has changed. To enable bitmap to speed up rebuilds and sync........
  • mdadm create RAID 1 array example


    mdadm --create /dev/md2 --level=1 --raid-devices=2 /dev/sda3 /dev/sdb3 cat /proc/mdstat Personalities : [raid1] md2 : active raid1 sdb3[1] sda3[0]       1363020736 blocks super 1.2 [2/2] [UU]       [=>...................]  resync =  8.3% (113597440/1363020736) finish=276.2min speed=75366K/sec  ........
  • Kernel panic - not syncing: Attempted to kill init! Pid: 1,comm: init Tained: G I------------- 2.6.32-358.el6.x86_64 #1 Call Trace: [] ? panic+0xa0/0x16f [] ? do_exit+0x862/0x870 [] ? fput+0x25/0


    Kernel panic - not syncing: Attempted to kill init! Pid: 1,comm: init Tained: G        I-------------    2.6.32-358.el6.x86_64 #1 Call Trace:  [<fffffff8150cfc8>] ? panic+0xa0/0x16f  [<fffffff81073ae2>] ? do_exit+0x862/0x870  [<fffffff81182885>] ? fput+0x25/0x30  [<fffffff81073b48>] ? do_group_exit+0x58/0xd0  [<fffffff81073bd7>] ? sys_exit_........
  • kvm cannot boot xen guest using lvm


    This booting error is because the Xen PV guest image uses the Xen kernel, this is not compatible with anything but a host running a Xen kernel. I did a kpartx -av virtual.img and then it created some partitions that showed up in fdisk. I mounted it and did a chroot into it and removed the xen kernel and installed a normal kernel but Xen still shows the same kernel in Grub (only the Xen one). This is strange but it seems like this Xen PV guest has some sort of hidden or........
  • Howto Copy/Backup/Restore All Firefox Settings/Preferences/Addons/Bookmarks to another computer/desktop


    I've used rsync again for this because I found it very simple, I've only excluded the Cache directory since it's not necessary and could be several GB in size. rsync -Phaz --exclude=Cache/* user@remotehost.com:/home/user/.mozilla/firefox/profile.default/* /home/user/.mozilla/firefox/ec1n9opl.default/ This is a great way once again to get going how you were on the old/remote computer fairly quickly.  It even restores all of your history in........
  • Thunderbird How-To Copy/Backup/Restore Accounts and Settings to Another Computer


    The best way is to use rsync, I've set it up so it doesn't copy unnecessary files, or at least ones I'm sure aren't needed. Here is the rsync command I used (adapt to your specific Thunderbird profile location): rsync -hazv user@remotehost.com:/home/user/.thunderbird/sbrer.default/* /home/user/.thunderbird/4nyb0.default/ --exclude=global* --exclude=Cache --exclude=ImapMail --exclude=Mail This is a great way to get your e-mail accounts going on a new c........
  • mdadm: /dev/sdb1 not large enough to join array solution


    mdadm --manage /dev/md1 --add /dev/sdb1 mdadm: metadata format 00.90 unknown, ignored. mdadm: metadata format 00.90 unknown, ignored. mdadm: metadata format 00.90 unknown, ignored. mdadm: /dev/sdb1 not large enough to join array md1's first primary member /dev/sda3 has 57394 cylinders while the /dev/sdb1 has 57393 (1 less cylinder) which is why it won't work.  fdisk -l /dev/sda3 Disk /dev/sda3: 47........
  • Firefox 11 solution for hidden tabs to the right


    After about 15 tabs I believe, they become hidden on the right side and are accessible only by furiously clicking the "right arrow" and even then it takes forever and honestly this hidden tab feature seems to slow down everything. If you enter "about:config" in your browser without the quotes just set this value to get rid of this annoying (one of many new) feature from Firefox (set it to false and restart your browser): services.sync.prefs.sync.brows........
  • HowTo Migrate and Import iPhone/Outlook Contacts into Android without using GMail using .vcf files


    I searched for days after getting my Galaxy Note and couldn't find a way to do this (at least not without buying programs for either Android/Windows). All I read was ways to sync and import the contacts to GMail but I don't want to use GMail for privacy reasons.  GMail/Google steal all of your personal information and use it for whatever purposes they want to and may sell or release it to who knows where (I don't care what their policy says but this stuff happens), just like the default........
  • Ubuntu 10.04 Flash Videos have tearing/lines Solution


    This is what I call a ridiculous bug in compiz that causes it. The solution/To fix it just install "compizconfig-settings-manager" Open it up and Navigate to General Options -> Display Settings Set the "Refresh Rate" to the current refresh rate of your monitor. For ATI cards with slow window dragging/response supposedly disabling "Sync to VBlank" fixes that but otherwise leave it as is.........
  • Linux Kernel Panic Messages - Symptoms of bad RAM module/stick


    These were caused by a bad stick of Corsair RAM [<c0466769>] free_hot_cold_page+0xfc/0x150 [<c04667d1>] __pagevec_free+0x14/0x1a [<c0468c6b>] release_pages+0x127/0x12f [<c04692d1>] __pagevec_release+0x15/0x1d [<c04697db>] __invalid_mapping_pages+0x120/0x156 [<c046981........
  • Openvz migration error solution Error: Failed to suspend container CPT ERR: f68cf000,25000 :foreign process 15755/14731(vzctl) inside CT (e.g. vzctl enter or vzctl exec). CPT ERR: f68cf000,25000 :suspend is impossible now.


    Starting online migration of CT 25000 to 192.168.5.1 Preparing remote node Initializing remote quota Syncing private Live migrating container... Error: Failed to suspend container CPT ERR: f68cf000,25000 :foreign process 15755/14731(vzctl) inside CT (e.g. vzctl enter or vzctl exec). CPT ERR: f68cf000,25000 :suspend is impossible now. CPT ERR: f68cf000,25000 :foreign process 15755/14731(vzctl) inside CT (e.g. vzctl enter or........
  • Linux Out of Memory OOM Object Killer Solution "Out of memory: kill process 1955 (sshd) score 81 or a child"


    I had a system running a 128MB live CD image with 2.8 gigs of available RAM and the OOM kernel killer went crazy when using dd for more than 8 minutes and kept killing everything.  I've read that this is due to a low-memory issue and paging in the kernel and 32-bit systems with lots of RAM. I even enabled swapspace on my LiveCD and the issue happened 25 minutes into dd rather than 8 minutes, so what gives? Also no swap space was ever used! cat /proc/s........
  • How To Test If Your Hard Drive Is Good/Signs of dying hard drive


    I like dd, although it only reads it, usually a read test of the entire disk will uncover if your hard drive is bad in some parts.  This is a good thing to do at least once a month, a lot of times bizarre program behavior, laginess and crashing/unnmounting problems etc.. are due to a failing disc and SMART won't know it or indicate a problem: We must also remember there's never a guarantee, I've found that ever since we moved to larger and more platters per drive with 1TB drives........
  • GRUB Solution to error "Error 6: Mismatched or corrupt version of stage1/stage2"


        GNU GRUB  version 0.97  (640K lower / 3072K upper memory)  [ Minimal BASH-like line editing is supported.  For the first word, TAB    lists possible command completions.  Anywhere else TAB lists the possible    completions of a device/filename.] grub> root (hd1,0)  Filesystem type is ext2fs, partition type 0xfd grub> setup........
  • How to recover from dead DRBD partition/hard drive in two simple commands


    This assumes that you've at least created the correct partition for your DRBD already. Notice that I am "diskless", that's because either your DRBD partition doesn't exist/has been renamed (eg. sdb becomes sda when sdb dies and you reboot) or because that drive is really actually dead/gone. *If you need to permanently change the partition/device for your resource be sure to edit /etc/drbd.conf on both hosts and reload the config. (replace r0 with........
  • Are my WDEARS20 Drives dying already?


    This happened during a RAID array check: SMART says both drives pass the test, but I'm doing a long test on them and hopefully this is not a hardware error. Apr  3 04:22:01 remote kernel: md: syncing RAID array md2 Apr  3 04:22:01 remote kernel: md: minimum _guaranteed_ reconstruction speed: 1000 KB/sec/disc. Apr  3 04:22:01 remote kernel: md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for reconstruction. Apr........
  • diff: memory exhausted


    I don't have a solution other than to use rsync, I used diff on about 1.7TB of data which includes hundreds of thousands if not millions of small files to ensure nothing was missing or corrupt. diff didn't even get past the first large directory without spitting that error out. Keep in mind I used "diff -r" because that means recursive, otherwise it wouldn't compare all files and subdirectories and would be a false way of doing it.........
  • Linux High IOWAIT updatedb can't be killed and crash with mdadm


    high IO wait   424 root      39  19  1900  848  552 D  0.0  0.0   0:00.91 updatedb root       424  0.0  0.0   1900   848 ?        DN   Mar11   0:00 /usr/bin/updatedb -f sysfs?rootfs?bdev?proc?cpuset?binfmt_misc?debugfs?sockfs?usbfs?pipefs?anon_inodefs?futexfs?tmpfs?inotifyfs?eventp........
  • CPU #1 not responding - cannot use it.


    I think this will be useful to others because I have a server that kept crashing mysteriously during intense disk usage/RAID checks.  It would only crash during the weekly RAID integrity check. Then I noticed during a reboot that not all CPUs were being brought up, as a result this actually creates much higher temperatures with the output I got from sensors, just booting the system produced higher than normal temperatures. You can imagine that a full blown RAID check........
  • rsync and cp doesn't/won't include .hidden files


    I was shocked that options like preserve and archive made no difference!  This is a big deal and will catch people off guard. Rsync include hidden files Solution: You need to use something like: rsync -Pha /source/dir/. /dest/dir *Notice the "." at the end of the source directory. cp -a still ignores them too, the solution is the same: cp -a /source/directory/. /destination........
  • updatedb/mdadm caused a kernel panic?


    Jan 16 04:02:03 centosbox syslogd 1.4.1: restart. Jan 16 04:07:34 centosbox kernel: INFO: task updatedb:20771 blocked for more than 300 seconds. Jan 16 04:07:34 centosbox kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Jan 16 04:07:34 centosbox kernel: updatedb      D F78BE050  6476 20771  20766                &n........
  • Ubuntu 9.04 Crash


    CPU/Kernel/MB/RAID problem? Jan  5 12:45:05 testbox kernel: [653298.890004] BUG: soft lockup - CPU#0 stuck for 61s! [hal-acl-tool:4168] Jan  5 12:45:05 testbox kernel: [653298.890005] Modules linked in: vmnet vmci vmmon binfmt_misc drbd video output input_polldev ocfs2_stackglue ocfs2_dlmfs ocfs2_dlm ocfs2_nodemanager configfs k8temp hwmon_vid lp snd_hda_intel snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_dummy snd_seq_oss snd_seq_midi snd_rawmidi........
  • Installing Webmin & Enabling SSL


    Webmin Setup Centos 5: wget http://downloads.sourceforge.net/project/webadmin/webmin/1.530/webmin-1.530-1.noarch.rpm?r=http%3A%2F%2Fwww.webmin.com%2Fstandard.html&ts=1294339690&use_mirror=surfnet [1] 24229 [2] 24230 [root@host ~]# --2011-01-06 21:48:20--  http://downloads.sourceforge.net/project/webadmin/webmin/1.530/webmin-1.530-1.noarch.rpm?r=http%3A%2F%2Fwww.webmin.com%2Fstandard.html Resolving downloads.sourceforge.net... 216.34.181.........
  • mdadm Centos 5 automatic resync


    This made me nervous but it's clearly a cronjob based on the messages log that happens every Sunday at about 4:22. I actually can't find any evidence of it in cron.d cron.daily but it is there somewhere obviously. What I don't get is why doesn't this cronjob do a datacheck like Ubuntu's cronscript does?  When you unnecessarily rebuild the array you lose your redundancy during that point which makes your data extremely vulnerable. *Update I did a grep of &q........
  • ntpdate[30630]: no servers can be used, exiting


    You're not supposed to call ntpdate like that, it's part of the server and needs a bunch of different arguments. The chances are if your time is out of sync and you installed ntpd, you need to start the ntpd service.........
  • mdadm/md-check how to check array integrity without rebuilding


    This doesn't seem to be widely known (maybe it's in some documentation that none of us read though) but there's an easy way to check the integrity of any mdadm array: sudo echo check > /sys/block/md0/md/sync_action -bash: /sys/block/md0/md/sync_action: Permission denied sudo will never work, this only works as root since echo is not actually a binary/command.  It is built-into bash. /sys/devices/virtu........
  • Cygwin and crontab backups via ssh/scp/rsync


    Install the "Editors" and "Net" groups that will give you rsync, ssh, ssh-keygen and cron. The trickiest thing that I keep forgetting about each time is you have to run "cron-config" which adds the cron service to Windows, and without doing that obviously no cron jobs will be run thus making automatic backups impossible. Warning about rsync/cygwin and using the -a archive switch. It's a good thing I caught this because it doesn't work ri........
  • rsync preserve attributes, permissions/ownership and times without being root non-root user


    One of the purposes of rsync is to backup whole filesystems and archive them but how can you do that properly and restore things to normal if all permissions and ownerships are not preserved from your root filesystem? It's not desirable to have everything running as root, especially not just for an rsync. The Easy rsync preserve permission solution for non-root users sudo is the answer and all you have to do is edit /etc/sudoers At the end o........
  • Ubuntu/Debian DRBD 8.0 Setup Guide


    I've only used it on Centos, so I thought I'd make a quick Debian guide: Install the DRBD Package apt-get install drbd8-utils Reading package lists... Done Building dependency tree       Reading state information... Done The following packages were automatically installed and are no longer required:   libswfdec-0.8-0 Use 'apt-get autoremove' to remove them. The following........
  • mdadm RAID 1 adventures


    I separated the 2 drives in the RAID 1 array. 1 is the old one /dev/sda and is out of date, while the separated other one /dev/sdc was in another drive and mounted and used with more data (updated). I wonder how mdadm will handle this: usb-storage: device scan complete md: md127 stopped. md: bind<sda3> md: md127: raid array is not clean -- starting background reconstruction raid1: raid set md127 active with 1 out of 2 m........
  • mdadm won't boot due to fsck error


    Nov 29 20:17:58 ubuntu kernel: [ 1157.180789] md: md1 stopped. Nov 29 20:17:58 ubuntu kernel: [ 1157.180829]  md0: unknown partition table filesystem not responding/reading properly with du or rsync (this needed an fsck). [ 2571.489217] EXT3-fs error (device md2): ext3_lookup: deleted inode referenced: 35923106 [ 2571.942299] EXT3-fs error (device md2): ext3_lookup: deleted inode referenced: 35923110 [ 2571.9568........
  • Ubuntu 10.04/GRUB2 mdadm won't boot


    Moving to RAID was a pain. What you have to do is the following from an existing install: Install mdadm Create your mdadm RAID 1 array on your spare hard drive. Start it with the missing disk. rsync the entire contents of your current / to the md partition. Here's a good way of doing it: rsync -Pha --exclude=/proc/* --exclude=/sys/* --exclude=/mnt/* /. /mnt/md2........
  • DRBD WFConnection Problem/Solution


    This has stumped me  a few times because I keep forgetting that Centos 5.5 comes with a default iptables configuration that ends up blocking DRBD traffic, I tried all the normal things and couldn't understand why I couldn't make my normal DRBD config work.  So if you have WFConnection problems and have tried the normal "mailing list" fixes, check your firewall status first! Both Nodes Say the Following: version: 8.3.8 (api:88/prot........
  • mdadm Linux Software RAID QuickStart Guide


      Create New RAID 1 Array: First setup your partitions (make sure they are exactly the same size) In my example I have sda3 and sdb3 which are 500GB in size. mdadm --create /dev/md2 --level=1 --raid-devices=2 /dev/sda3 /dev/sdb3 mdadm: array /dev/md2 started. Check Status Of The Array *Note I already have other arrays md0 and md1. You can see below that md2 is syn........
  • md: kicking non-fresh sda1 from array! - fix/solution


    md: Autodetecting RAID arrays. md: autorun ... md: considering sdb1 ... md:  adding sdb1 ... md:  adding sda1 ... md: created md0 md: bind<sda1> md: bind<sdb1> md: running: <sdb1><sda1> md: kicking non-fresh sda1 from array! md: unbind<sda1> md: export_rdev(sda1) raid1: raid set md0 active with 1 out of 2 mirrors The md0 raid kicked sda1 ou........
  • mdadm "auto-read-only" Linux Software RAID solution


    If you have the "(auto-read-only)" beside an array I have no idea why that happens but it is easy to fix. Just run "mdadm --readwrite /dev/md1" (rename md0 to the device with the problem and it will begin to resync.   md1 : active (auto-read-only) raid1 sdb2[0] sda2[1]       19534976 blocks [2/2] [UU]         resync=PENDING      ........
  • Realtek r8169 1gbit/1000mbit NIC/Ethernet Goes Down/Halts on Large File Transfer/Heavy Network Usage


    This is obviously a bug in the r8169 kernel module and it seems to affect a lot of people.  I upgraded to the latest kernel and hope this won't happen anymore, as it is a very serious error.  This is especially serious for those who are running servers with this chipset, who can afford for the NIC to randomly go off-line for no apparent reason?   [655548.189113] type=1505 audit(1277067560.902:5): operation="profile_load" name="/usr/bin/freshclam&q........
  • rsync skipping non-regular file


    skipping non-regular file "mnt/lib/udev/devices/console" skipping non-regular file "mnt/lib/udev/devices/kmem" skipping non-regular file "mnt/lib/udev/devices/loop0" skipping non-regular file "mnt/lib/udev/devices/null" skipping non-regular file "mnt/lib/udev/devices/ppp" skipping non-regular file "mnt/lib/udev/devices/net/tu Solution for skipping non-regular file You need the -D........
  • Why does mdadm have two sets of UUID's?


    Which one does the OS care about?  blkid says the UUID is "787f1fa4-b010-4d77-a010-795b42884f56" while md insists its UUID is "4d96dd3b:deb5d555:7adb93cb:ce9182d9" When in doubt, do we assume the OS takes the one from blkid?  /dev/md0: UUID="787f1fa4-b010-4d77-a010-795b42884f56" TYPE="ext3" [root@localhost ~]# mdadm -D /dev/md0 /dev/md0:         Version : 0.90 &........
  • Centos 5.5 RAID 1 boot issue


    I installed 5.5 with a 300GB RAID 1 partition (boot is also on this partition).  It booted up fine the first few times until after I used a Live CD and accessed the array, and it became named /dev/md127 for some reason. Now whenI boot into CentOS I get a kernel panic and different errors, once I got "invalid superblock", even though the array is fine (it didn't happen again, probably because I was sure to dismount and stop the mdadm array properly). Here's what........
  • mdadm Linux Software RAID auto-detect and assemble RAID Array


    mdadm --assemble --scan mdadm: /dev/md/diaghost05102010:2 has been started with 2 drives. mdadm: /dev/md/diaghost05102010:1 has been started with 2 drives. mdadm: /dev/md/diaghost05102010:0 has been started with 2 drives. -bash-3.1# cat /proc/mdstat Personalities : [linear] [raid0] [raid1] [raid6] [raid5] [raid4] [multipath] md125 : active raid1 sda1[0] sdb1[1]       14658185 blocks super 1.2........
  • rsync halted froze Out of memory: kill process 7559 (rsync) score 635 or a child Killed process 7559 (rsync)


    Out of memory: kill process 7559 (rsync) score 635 or a child Killed process 7559 (rsync) I was surprised to see this in my dmesg when my rsync backup suddenly stalled/stopped. This system has 3 gigs of RAM and lots of free memory so I don't understand what is happening.   rsync invoked oom-killer: gfp_mask=0x200d2, order=0, oomkilladj=0 Pid: 7600, comm: rsync Not tainted 2.6.24.2 #83  [<c014990e>] oom_kill_pr........
  • rsync my favorite/best/optimal default switches and options


    Not sure what rsync switches/options to use? rsync -PDrphogtl The short version would be: rsync -Pha I think these are really common sense options to use and probaby should be the default. Explanation of rsync switches P = display the progress D = hybrid of --specials and --devices so all special and device files will be copied as well. r = recursive (otherwise rsync won't copy files deeper than........
  • rsync ERROR: sdb2/backup.r18 failed verification -- update retained.


    ERROR: sdb2/backup.r18 failed verification -- update retained. This happened when I was rsyncing a dying hard drive, a lot of people seem to report this as an error with rsync itself but I don't think rsync is the issue at all. I know the source hard drive was bad, it was having all kinds of timeout, seek, hardware ECC recovered and reset errors. The only time something else could be the problem is if the data is being corrupted in transit, perhaps through........
  • Convert HDD/Hard Drive Partition(s) into non-RAID into RAID 1 using existing data without data loss and without reformatting.


    Before we start I take no responsibility for this, you should have a backup and if you make a mistake during this process you could wipe out all of your data.  So backup somewhere else before starting this as a precaution, or make sure it's data you could afford to lose. The RAID 1 Setup (Hardware Wise) I've already setup my 2 x 1TB (Seagate) drives with identical partitions, make sure your new hard drive (the empty one) is setup like your curr........
  • Linux EXT3 16GB-17GB maximum filesize issue solved/how to fix


    I have no idea why but mkfs.ext3 defaults to a patheticlly small blocksize of 1024 bytes/1KB (kilobyte).  That means the maximum filesize is ONLY 16GB!  With 2KB/2048 bytes you get a 256 GB maximum filesize, and with 4KB/4096 bytes you get 2TB! I finally noticed/paid attention to this after realizing that with rsync and scp that no file larger than 17GB could be transferred.  I then realized it must be a file size limit on the partition. Here is what tune2fs tol........
  • tar gzip: stdout: File too large - Error Size Limit on Large file > 17GB


    The folder I was trying to archive is about 72GB, but much like rsync at about 17GB it chokes because of the filesize.  What's with so many common and essential Linux tools having such limitations?  I guess it is likely that the authors never wrote their code with the idea that files would be so large but it's still very annoying.  It's important to stay on top of these limitations on production servers because I didn't realize what happened until I checked the file with "........
  • Rsync Timeout/Stall On Large files bigger than 16GB/Cannot Transfer large files with rysnc


    The last transfer I was able to do was 16.77GB according to rsync's output and it worked fine.        16.77G 100%   10.66MB/s    0:25:00 (xfer#17484, to-check=234753/256249) But on a file over 17GB (gigs) it stalls.  Trust me, it really is stalled, it's been about 8 hours with no progression from that point for some reason. 17.24G  54%   10.65MB/s    0:22:07........
  • Picking an FTPD (vsftpd) Server in Linux Centos/Debian


    I decided on using yum to help me decide even though I normaly use proftpd I decided to see what else I could find. yum search ftp Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile  * rpmforge: ftp-stud.fht-esslingen.de  * base: mirrors.netdna.com  * updates: updates.interworx.info  * addons: yum.singlehop.com  * extras: mirrors.netdna.com rpmforge   ........
  • PHP cannot access /usr/bin/openssl


    PHP cannot access /usr/bin/opensslI have verified the username that runs the process is able to access /usr/bin/openssl and it does exist but the PHP script is saying it doesn't exist: [code:1:1fd0f3abbe] if (!file_exists($OPENSSL)) { //echo "ERROR: OPENSSL $OPENSSL not foundn"; }[/code:1:1fd0f3abbe] I don't get itI can clearly see the contents of /usr/bin by using the PHP system fu........
  • rsync bash script


    rsync bash script [code:1:722d8a25c1]#!/bin/bash # config --------------------------------- # two methods # from = receive data from another server # to = send data to another server rsync_method=from rsync_ip='192.168.5.18' local_dir='/home/backupguy/backups' remote_dir='/home/backup' free_space_bin='/home/backups/freediskspace.sh' # config end ------------------------------ if [ '$rsync_me........
  • Basic Port Listing


    Basic Port ListingHopefully someone finds this useful or at least interesting. http://www.sans.org/top20/#u9 Name Port Protocol Description Small services <20 tcp/udp small services FTP 21 tcp file transfer SSH 22 tcp login service TELNET 23 tcp login service SMTP 25 tcp mail TIME 37 tcp/udp time synchronization WINS 42 tcp/udp WINS replication DNS 53 udp naming service........
  • iPhone/iTunes sync error "0xE8008001"


    This happened during a sync of my iPhone while installing some apps.  This usually happens to non-jailbroken iPhones who are trying to install non-Apple approved software OR jailbroken phones who haven't done the "Sync update". How to fix this error and get the Sync Update Open Cydia - Manage - Sources - Edit Add this source: http://cydia.hackulo.us Then under the section "Hackulo.us" install "Ap........
  • OCFS2 crash


    When trying to even cd or ls the mounted OCFS2 partition it crashes.  I think this is a combination of VMWare Server's problem and the way I mounted and symlinked to it. More than anything this shows the problem and lack of forsight with VMWare, but also that OCFS2 is easily crashed if you do strange things. Output of /var/log/messages for OCFS2   Apr 10 15:57:45 localhost kernel: [84331.691258] Modules linked in: vmnet vmci vmmon ocfs2_stac........
  • Samba SMB Error - Server not using user level security and no password supplied. tree connect failed: NT_STATUS_WRONG_PASSWORD


    Server not using user level security and no password supplied. tree connect failed: NT_STATUS_WRONG_PASSWORD That happens when trying to use smbclient to connect to a share.  The weird thing is that I can authnenticate just fine from Windows XP. It is partially my mistake, I forgot this share does have a password.  I've tried authenticating with the correct user and also with "Guest" because this works in Windows.  In Linux I ........
  • Latest Articles

  • syntax error, unexpected T_SL in PHP Solution
  • grep regular expression match number range between specific numbers
  • bash how to print out lines of text within a range from the first occurrence
  • bash script how to to check LAN computers for open ports
  • MySQL Using mytop Debug Source of High IO and Slow Performance
  • How To Mathematically Convert and Calculate Binary Value To Decimal Value
  • systemd management using systemctl and journalctl to check systemd logs
  • css how to format code in the code tag
  • css br with extra blank line
  • What is /dev/pts and why do we need it in Linux?
  • Linux What is umask (user mask) for file and directory creation permissions and how to calculate umask and change the defaults
  • Linux Permissions and Groups Real Usecase for Group Access
  • Centos 7 python34 how to install the gi library
  • Centos 7 - Convert Minimal to Graphical GUI GNOME or KDE Desktop
  • AMD Set Fan Speed and Other Powerplay Memory/CPU Timings with a Linux script
  • Ethereum Mining Claymore Nanopool Error
  • genisoimage errors with long filenames and deep directory structures
  • Linux Kernel USB Export Errors
  • How to download gajim 0.16.9 XMPP/Jabber client so you can use OMEMO encryption
  • HP DL385 G7 Linux BIOS Update Flash