Server Administration
BIND/NAMED woes
Feb 5 01:39:33 server named[19768]: zone myzone.com/IN: serial number (12331465) received from master 127.0.0.2#53 < ours (200901281) The above is taken from /var/log/messages This can be annoying, it can happen for a variety of reasons. What seems to be happening here is that the slave realizes the time on the slave is ahead of the master, so it therefore assumes it has the most up to date copy and won't actually transfer the zone. The solution is easy. Here's what you type on the slave: rndc retransfer myzone.com *Replace "myzone.com" with the actual name of the domain you want to transfer to the slave (of course the slave must already have the domain defined in /etc/named.conf. Remember that "myzone.com" is the domain name, even if you have a zone file called "myzone.com.db" you wouldn't type that or it won't work. The command above forces BIND to transfer the zone no matter what.
Apache Ignoring/Not processing .htaccess file
I was getting very frustrated one day wondering why it appeared my .htaccess file was being ignored and not processed by Apache. No matter what I did it was obvious that Apache didn't care about my .htaccess file. Then I realized that the default settings must be in effect, which is that my vhost didn't explicitly allow me to override the default settings. This usually comes down to your vhost settings. Make sure you have an entry like this in your Apache vhost settings in order for .htaccess files to be processed, otherwise the default .htaccess settings in /etc/httpd/conf/httpd.conf are what take effect. Options FollowSymLinks AllowOverride All As you can see above, you just need an entry like this to fix the problem. Be sure that the path in the
MySQL Restoring And Dumping/Backing UP MySQL Data/Tables/Databases
Backing MySQL Databases Backing Up/Dumping All Mysql Databases To A Single File mysqldump --all-databases -u admin -p > allmysqldatabases.sql The "-all-databases" clause is pretty obvious isn't it? It means that it will backup all databases. The "-u admin" means login using the user "admin", if you have another user such as root or a specific user that can access/dump all databases you can use that one. The ">allmysqldatabases.sql" part tells it to output all the database information to a file called "allmysqldatabases.sql" and don't forget the ">" as that is redirection and is what writes all the info to a file. Without redirecting the output, you'll see lines and lines of gibberish on your screen, because otherwise MySQL assumes you want to print all the output to the screen. This method is a quick and dirty way to migrate several databases at once. After that if you want to get fancy, you can gzip your database dump, this will protect against unknown data corruption and of course make the file transfer quicker since compressing with gzip reduces the size. For example the file was originally 279MB and gzip took it to 179MB. To gzip your backed up mysql databases just run this command (works on any file other than just .sql dumps of course): gzip allmysqldatabases.sql and it leaves you with "allmysqldatabases.sql.gz" in compressed gzip format. Backing Up A Single Mysql Database mysqldump --databases mysql -u admin -p > mysqlbackup.sql The "--databases" clause specifies you are going to backup one or more database as specified in the command. If you wanted databases "mysql" and "someothedb" you would use "--databases mysql someotherdb" Backing Up A Specific Table mysqldump -u user -p databasename tablename tablename > thetable.sql As you can see after specifying the db name you can specific as many tables you want. By default the tables will be populated with the data. If you just want the structure use this instead. mysqldump -d -u user -p databasename tablename tablename > thetable.sql The "-d" switch causes only the structure to be dumped. Restoring MySQL Databases What fun is backing up without being able to restore? Restoring can be a bit more tricky and there's a few different scenarios and options you have. I'll focus on the most common one, a new MySQL Server install with no previous data or tables. I'm assuming you've transferred your database backup "allmysqldatabases.sql.gz" To restore all your database(s) and information to MySQL just run this following command: mysql -u root < allmysqldatabases.sql Restoring a specific MySQL database/script backup to a specific database mysql -u $username --password="$password" --database="$dbname" < $sqlscript Issues/Problems/Complications with Restoring MySQLDatabases One issue I have is that despite backing up and restoring the database "mysql" I cannot authenticate successfully with my old login information. I don't know why because a query of the mysql table shows the usernames I am expecting to see. I'll have to research more on this point, but the good news is that all the other tables/databases and entries are there as expected. *UPDATE Iknew everything was there and that the "mysql" table was fully populated. All I had to do was restart mysql server (mysqld) and everything worked as normal with the old passwords and privileges Ioriginally had.
Linux Bash Shell Cannot/Can't View File That Starts With - Dash ?
Iwas getting really annoyed with this, I used full quotes around the filename and vi, cat, less all thought I was trying to pass the dash in the filename as an argument. I didn't realize that all you have to do is just put a dash dash "--"in front. Here's an example:cat -- "-etc-glusterfs-glusterfsd.vol.log" Without the -- you get: cat: invalid option -- c Try `cat --help' for more information. Ihope this saves someone else some trouble. I never thought that a dash of anything could fix such an annoying problem :) I still think it's not preferable to have a filename which starts with a - but gusterfs thinks it is a smart thing for now.
Mysqld Solution - Can't init databases /usr/libexec/mysqld: Can't read dir of '/tmp/' (Errcode: 13) /usr/libexec/mysqld: Can't create/write to file '/tmp/ibyP1qUC' (Errcode: 13) 100215 7:02:24 InnoDB: Error: unable to create temporary file; errno
100215 07:02:24 mysqld started /usr/libexec/mysqld: Can't read dir of '/tmp/' (Errcode: 13) /usr/libexec/mysqld: Can't create/write to file '/tmp/ibyP1qUC' (Errcode: 13) 100215 7:02:24 InnoDB: Error: unable to create temporary file; errno: 13 100215 7:02:24 [ERROR] Can't init databases 100215 7:02:24 [ERROR] Aborting 100215 7:02:24 [Note] /usr/libexec/mysqld: Shutdown complete 100215 07:02:24 mysqld ended One of my servers was rebooted for some reason and all sites and e-mail accounts were down because the database could not be connected. The first thing Ichecked was mysqld of course and it was not running. The above was revealed in /var/log/mysqld.log I was starting to panic and wondered if the server was compromised and I even checked /var/lib/mysql to make sure all the database data was still there. The solution for this MySQL Error is simple: chown root.root /tmp chmod 1777 /tmp And of course be sure that you start MySQL again.
Create/Enable SSL Certificates for Apache on Linux/Unix Systems eg. Redhat,Centos,Debian
Shortcut/Easiest Way To Create A Self-Signed Key: openssl req -new -x509 -nodes -days 1530 -out server.crt -keyout server.key Using the above, you instantly create a self-signed certificate valid for 1530 days and you can simply skip to step #5.) below. If You Need a Real SSLCertificate (eg. Equifax/Openssl) then you need to create a CSR request (you'll need to follow Steps 1.) and 2.) in order to create the CSR. You then upload the CSR Certificate to your SSLProvider and they will e-mail you the .crt (SSL Certificate). Remember to keep your key from Step 1, you need the .key and .crt (from your SSL) provider to make use of your certificate. 1.) Create Your Private Key openssl genrsa -out ssl-private.key 2048 (note above I didn't include the -des3 switch as that makes you choose a passphrase for your key which we don't want). *Be sure not to lose this key file, especially if you are using the CSR request in Step 2.) to apply for a real SSL certificate Generating RSA private key, 2048 bit long modulus .............................................+++ ..................................................................................+++ e is 65537 (0x10001) Enter pass phrase for ssl-private.key: Verifying - Enter pass phrase for ssl-private.key: 2.) Create CSR Request (Certificate Signing Request) openssl req -new -key ssl-private.key -out ssl-request.csr Enter pass phrase for ssl-private.key: You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Country Name (2 letter code) [GB]: State or Province Name (full name) [Berkshire]: Locality Name (eg, city) [Newbury]: Organization Name (eg, company) [My Company Ltd]: Organizational Unit Name (eg, section) []: Common Name (eg, your name or your server's hostname) []: Email Address []: Please enter the following 'extra' attributes to be sent with your certificate request A challenge password []: An optional company name []: 3.) Remove Password/Passphrase From Private Key (otherwise Apache asks for the password each time you start it) mv ssl-private.key ssl-private.key-pass openssl rsa -in ssl-private.key-pass -out ssl-private.key Enter pass phrase for ssl-private.key-pass: writing RSA key 4.) Make Self Signed Certificate openssl x509 -req -days 730 -in ssl-request.csr -signkey ssl-private.key -out ssl-certificate.crt unable to load certificate 20243:error:0906D06C:PEM routines:PEM_read_bio:no start line:pem_lib.c:632:Expecting: TRUSTED CERTIFICATE *(don't forget the -req or you'll get the above error) 5.) Place CRT & KEY inside /etc/httpd/conf/httpd.conf or edit your VHOST conf cp ssl-private.key ssl.key/server.key cp ssl-certificate.crt ssl.crt/server.crt Here is what you should be inside the vhost (obviously be sure that you have the correct path and filenames): SSLEngine on SSLVerifyClient none SSLCertificateFile /etc/httpd/ssl-certs/server.crt SSLCertificateKeyFile /etc/httpdssl-certs/server.key *Remember to restart Apache or the new certificate won't be applied/take effect until you do.
Apache/Mod_SSL not serving the right/expected certificate?
There is actually by default a "Default SSL" vhost that can mess things up for you and can cause surprising and unexpected results. Default Apache SSL Cert in /etc/httpd/conf.d/ssl.conf there is a default SSL Virtual Host which screws things up by offering itself instead of the SSL cert I specify in my own vhosts
Skype For Business 2015 and 2019 Guide, Reference, Howto and Troubleshooting Solutions
https://docs.microsoft.com/en-us/powershell/module/skype/?view=skype-ps Troubleshooting Client Connectivity Issues The first step is to enable full logging in the Skype Client itself and then check the log on the client side. In Windows this normally means going to the Skype client "settings" and then "Enable Full Logging". After that you can open Event Viewer, click on the Application Log and see what entries for "Lync" come up (yes it is known as Lync despite the fact the product itself is now called Skype 2015 or 2019 for business). Further Troubleshooting on the Front End Server Pool or Edge Server Pool The first step here is to really just check th "Lync Server" log under "Event Viewer" and the "Applications and Services Logs"
QEMU-KVM won't boot Windows 2016 or 2019 server on an Intel Core i3
CPU:Intel(R) Core(TM) i3-2120 CPU @ 3.30GHz MOBO: Manufacturer: ASUSTeK COMPUTER INC. Product Name: P8H61-M LX3 PLUS R2.0 qemu-kvm-0.12.1.2-2.506.el6_10.1.x86_64 This is weird but the only OS I've found this machine doesn't work with is Windows 2019 Server. Ihave no idea, when 2008, 2012 work fine. Windows 2019 also works with the same software (KVMversion) on a different MOBOand CPU, so I suspect it is something CPU or MOBOrelated that is not playing nicely. Solution: Windows 2016+ (eg 2019) will NOT boot without using the "-cpu host"parameter which passes through the host CPU. On most machines I run, especially server hardware this doesn't seem to matter (eg. I normally just use the default QEMU-CPU and all is fine even on 2019 and 2016). Here is an example: qemu-system-x86_64 --enable-kvm -cpu host -smp 8 -m 8192 -drive format=raw,file=the-file.img When booting my Windows 2019 template all Iget is the Windows logo:
How to install and configure haproxy on Linux Ubuntu Debian
haproxy is one of the best known and widely used Open Source load balancers out there and a strong competitor to nginx. haproxy is used by many large sites per Wikipedia: HAProxy is used by a number of high-profile websites including GoDaddy, GitHub, Bitbucket,[6] Stack Overflow,[7] Reddit, Slack,[8] Speedtest.net, Tumblr, Twitter[9][10] and Tuenti[11] and is used in the OpsWorks product from Amazon Web Services.[12] According to some stats data haproxy is even more popular than the AWS Elastic Load Balancer: Step 1 - Install apt install haproxy Reading package lists... Done Building dependency tree Reading state information... Done The following packages were automatically installed and are no longer required: acl ebtables galera-3 git git-man iproute2 less libatm1 libconfig-inifiles-perl libdbd-mysql-perl libdbi-perl liberror-perl libjemalloc1 liblzo2-2 libuv1 lsof mariadb-common netcat netcat-traditional patch pigz runc socat squashfs-tools ubuntu-fan xdelta3 Use 'apt autoremove' to remove them. Suggested packages: vim-haproxy haproxy-doc The following NEW packages will be installed: haproxy 0 upgraded, 1 newly installed, 0 to remove and 34 not upgraded. Need to get 1116 kB of archives. After this operation, 2374 kB of additional disk space will be used. Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 haproxy amd64 1.8.8-1ubuntu0.13 [1116 kB] Fetched 1116 kB in 2s (657 kB/s) perl: warning: Setting locale failed. perl: warning: Please check that your locale settings: LANGUAGE = (unset), LC_ALL = (unset), LANG = "C.UTF-8" are supported and installed on your system. perl: warning: Falling back to the standard locale ("C"). locale: Cannot set LC_CTYPE to default locale: No such file or directory locale: Cannot set LC_MESSAGES to default locale: No such file or directory locale: Cannot set LC_ALL to default locale: No such file or directory Selecting previously unselected package haproxy. (Reading database ... 20143 files and directories currently installed.) Preparing to unpack .../haproxy_1.8.8-1ubuntu0.13_amd64.deb ... Unpacking haproxy (1.8.8-1ubuntu0.13) ... Setting up haproxy (1.8.8-1ubuntu0.13) ... Created symlink /etc/systemd/system/multi-user.target.wants/haproxy.service → /lib/systemd/system/haproxy.service. invoke-rc.d: could not determine current runlevel invoke-rc.d: WARNING: No init system and policy-rc.d missing! Defaulting to block. Processing triggers for systemd (237-3ubuntu10.57) ... Step 2 - Configure haproxy.cfg file vi /etc/haproxy/haproxy.cfg Here is how the defaults of haproxy.cfg look: global log /dev/log local0 log /dev/log local1 notice chroot /var/lib/haproxy stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd listeners stats timeout 30s user haproxy group haproxy daemon # Default SSL material locations ca-base /etc/ssl/certs crt-base /etc/ssl/private # Default ciphers to use on SSL-enabled listening sockets. # For more information, see ciphers(1SSL). This list is from: # https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/ # An alternative list with additional directives can be obtained from # https://mozilla.github.io/server-side-tls/ssl-config-generator/?server=haproxy ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS ssl-default-bind-options no-sslv3 defaults log global mode http option httplog option dontlognull timeout connect 5000 timeout client 50000 timeout server 50000 errorfile 400 /etc/haproxy/errors/400.http errorfile 403 /etc/haproxy/errors/403.http errorfile 408 /etc/haproxy/errors/408.http errorfile 500 /etc/haproxy/errors/500.http errorfile 502 /etc/haproxy/errors/502.http errorfile 503 /etc/haproxy/errors/503.http errorfile 504 /etc/haproxy/errors/504.http More info about configuring haproxy from the authors. Let's add a frontend and backend At the moment the load balancer does nothing and essentially has no usable configuration. We're going to add a frontend that listens on localhost and is bound to port 8080. The frontend itself is just the entry point for the user, the frontend is configured on a certain IPand port that we define and the next step is that we'll have to define a "backend" that is the actual source server (eg. our Apache running PHP or another application) Add this frontend and backend config to the end of haproxy.cfg frontend rttfrontend bind 0.0.0.0:8080 default_backend rttbackendservers backend rttbackendservers server backendserver01 127.0.0.1:80 cache rttcache # Total size of the cache in MB total-max-size 500 # Max size of any single item in bytes max-object-size 100000 # Time to live for each item in seconds # This can be overridden with a Cache-Control header max-age 3000 This config allows you to scale out as much as you need, for example you could add dozens or hundreds of backend servers with different IPs and ports. You may also want to add the "check" option after each server so requests won't be sent to dead or overloaded servers: server rttbackendserver01 server.com:9000 check We can make it more like a CDNby enabling cache, so the backend servers don't need to be contacted if we have a cache hit: cache rttcache # Total size of the cache in MB total-max-size 500 # Max size of any single item in bytes max-object-size 10000 # Time to live for each item in seconds # This can be overridden with a Cache-Control header max-age 3000 In older versions like 1.8, the max-object-size option does not exist. You'll find the cache doesn't work unless you set this option in your global config: tune.bufsize 9999999 Here is an example of how much performance can be gained by using a caching frontend haproxy server: In our first example below the page in question has not been cached and has a TTFB of 0.486955 seconds and total load time of .677587 seconds. curl -k -o /dev/null -w "Connect: %{time_connect} TTFB: %{time_starttransfer} Total time: %{time_total} n" $site % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 16413 100 16413 0 0 24243 0 --:--:-- --:--:-- --:--:-- 24207 Connect: 0.047765 TTFB: 0.486955 Total time: 0.677587 Now after we loaded the site and it is in the cache notice the difference in performance: TTFB is now 0.090424 and total load time of .135752 TTFB is now 5.38X faster and load time was 4.99X faster! curl -k -o /dev/null -w "Connect: %{time_connect} TTFB: %{time_starttransfer} Total time: %{time_total} n" $site % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 16413 100 16413 0 0 118k 0 --:--:-- --:--:-- --:--:-- 118k Connect: 0.044437 TTFB: 0.090424 Total time: 0.135752 How To enable Stats By enabling stats we can check on things like how our cache is doing: Add this to the globa section: stats socket ipv4@127.0.0.1:9999 level admin stats socket /var/run/hapee-lb.sock mode 666 level admin You can echo commands via socat to see the status of things like you cache: echo "show cache" | socat stdio /var/run/hapee-lb.sock 0x7f5f1ef9503a: rtt (shctx:0x7f5f1ef95000, available blocks:512000) 0x7f5f1ef950ac hash:3598866029 size:16657 (17 blocks), refcount:0, expire:25695 Here is a list of commands that can be sent: echo "help" | socat stdio /var/run/hapee-lb.sock Unknown command. Please enter one of the following commands only : help : this message prompt : toggle interactive mode with prompt quit : disconnect show tls-keys [id|*]: show tls keys references or dump tls ticket keys when id specified set ssl tls-key [id|keyfile] : set the next TLS key for the or listener to show errors : report last request and response errors for each proxy disable agent : disable agent checks (use 'set server' instead) disable health : disable health checks (use 'set server' instead) disable server : disable a server for maintenance (use 'set server' instead) enable agent : enable agent checks (use 'set server' instead) enable health : enable health checks (use 'set server' instead) enable server : enable a disabled server (use 'set server' instead) set maxconn server : change a server's maxconn setting set server : change a server's state, weight or address get weight : report a server's current weight set weight : change a server's weight (deprecated) show sess [id] : report the list of current sessions or dump this session shutdown session : kill a specific session shutdown sessions server : kill sessions on a server clear table : remove an entry from a table set table [id] : update or create a table entry's data show table [id]: report table usage stats or dump this table's contents clear counters : clear max statistics counters (add 'all' for all counters) show info : report information about the running process show stat : report counters for each proxy and server show schema json : report schema used for stats show startup-logs : report logs emitted during HAProxy startup show resolvers [id]: dumps counters from all resolvers section and associated name servers set maxconn global : change the per-process maxconn setting set rate-limit : change a rate limiting value set severity-output [none|number|string] : set presence of severity level in feedback information set timeout : change a timeout setting show env [var] : dump environment variables known to the process show cli sockets : dump list of cli sockets show fd [num] : dump list of file descriptors in use show activity : show per-thread activity stats (for support/developers) disable frontend : temporarily disable specific frontend enable frontend : re-enable specific frontend set maxconn frontend : change a frontend's maxconn setting show servers state [id]: dump volatile server information (for backend ) show backend : list backends in the current running config shutdown frontend : stop a specific frontend set dynamic-cookie-key backend : change a backend secret key for dynamic cookies enable dynamic-cookie backend : enable dynamic cookies on a specific backend disable dynamic-cookie backend : disable dynamic cookies on a specific backend show cache : show cache status add acl : add acl entry clear acl : clear the content of this acl del acl : delete acl entry get acl : report the patterns matching a sample for an ACL show acl [id] : report available acls or dump an acl's contents add map : add map entry clear map : clear the content of this map del map : delete map entry get map : report the keys and values matching a sample for a map set map : modify map entry show map [id] : report available maps or dump a map's contents show pools : report information about the memory pools usage