Reverse DNS does not match SMTP Banner

UPDATE: WHM/cPanel removed support for in version 11.50, so changes below are not valid for versions 11.50+

https://documentation.cpanel.net/display/ALD/11.50+Release+Notes#id-11.50ReleaseNotes-/etc/mail_reverse_dnsremoved

 

If you make an SMTP test on http://mxtoolbox.com you might be getting a following error shown in the test results “Reverse DNS does not match SMTP Banner”.

This error is showing because your SMTP greeting message is not matching the PTR records for the IP of the SMTP server used in test.

Following files need to be used and configured properly, for SMTP banner to match reverse DNS records.

/etc/mailhelo
/etc/mailips
/etc/mail_reverse_dns
/etc/exim.conf

Configure Exim to use mailhelo and mailips file

Go to WHM to Home »Service Configuration »Exim Configuration Manager and in Basic Editor on Domains and IPs tab set following settings:

Send mail from account’s dedicated IP address: OFF
Reference /etc/mailhelo for outgoing SMTP HELO: ON
Reference /etc/mailips for outgoing SMTP connections: ON

Configure necessary values in configuration files

Edit or create  /etc/mail_reverse_dns file and set the following in it for needed IPs.

x.x.x.x: rdns of the IP x.x.x.x
y.y.y.y: rdns of IP y.y.y.y

Edit or create /etc/mailhelo file and set following in it for the domains that you want to setup SMTP banner for.

example.com: reverse dns of the IP used for example.com domain
*: default SMTP HELO for unconfigured domains

Edit or create /etc/mailips file and set following in it:

example.com: x.x.x.x #x.x.x.x is the IP used for outgoing mail for domain example.com
*: y.y.y.y #y.y.y.y is the default IP that will be used for unconfigured domains

Configure exim.conf to use correct SMTP Banner

Following values need to be configured in exim.conf for SMTP Banner to be set to rDNS values set in /etc/mail_reverse_dns.

smtp_active_hostname
message_id_header_domain
smtp_banner

Be default only smtp_banner is set on cPanel servers, and it has a different value then needed.

root@server1 [~]# egrep "smtp_active_hostname|message_id_header_domain|smtp_banner" /etc/exim.conf
smtp_banner = "${primary_hostname} ESMTP Exim ${version_number} \

smtp_banner will probably look like this on your cPanel server.

"${primary_hostname} ESMTP Exim ${version_number}  \#${compile_number} ${tod_full} \n   We do not authorize the use of this system to transport unsolicited, \n   and/or bulk e-mail."
Configure values in exim.conf over shell

Locate the line smtp_banner and change its value so it looks like following:

smtp_banner = "${smtp_active_hostname} ESMTP Exim ${version_number} \"

Add smtp_active_hostname line value to exim.conf to look line following:

smtp_active_hostname = ${lookup{$interface_address}lsearch{/etc/mail_reverse_dns}{$value}{$primary_hostname}}

Add message_id_header_domain line to exim.conf to look like following:

message_id_header_domain = $smtp_active_hostname

In the end related values in exim.conf should look like this:

root@server1 [~]# egrep "smtp_active_hostname|message_id_header_domain|smtp_banner" /etc/exim.conf
smtp_banner = "${smtp_active_hostname} ESMTP Exim ${version_number} \"
smtp_active_hostname = ${lookup{$interface_address}lsearch{/etc/mail_reverse_dns}{$value}{$primary_hostname}}
message_id_header_domain = $smtp_active_hostname

Restart exim with /scripts/restartsrv_exim and SMTP tests should now pass without the SMTP banner warning.

Configure values in exim.conf over WHM

In your WHM go to Home »Service Configuration »Exim Configuration Manager and go to Advanced Editor.

Search for the smtp_banner field and change default value to:

"${smtp_active_hostname} ESMTP Exim ${version_number} \"

 

Edit smtp_banner in WHM
Edit smtp_banner in WHM

Find the “Add additional configuration setting” button and add two new configuration settings smtp_active_hostname and message_id_header_domain.

additional configuration settings
Add additional configuration setting in WHM

For smtp_active_hostname set the following value:

${lookup{$interface_address}lsearch{/etc/mail_reverse_dns}{$value}{$primary_hostname}}

For message_id_header_domain set the following value:

$smtp_active_hostname

References:

https://forums.cpanel.net/threads/easy-fix-your-smtp-banner-smtp-greeting-and-reverse-dns-for-dedicated-ips.391311/

https://forums.cpanel.net/threads/exim-banner-mail-headers-and-resellers-with-own-ip.100697/

How to provide SSH password inside a script or oneliner

If you ever need to provide a password for SSH login inside a bash script or a shell command, to avoid being asked a password when SSH keys are not used, it can be done with usage of expect command, or sshpass utility.

Using expect

Expect is a program that “talks” to other interactive programs according to a script.

http://www.tcl.tk/man/expect5.31/expect.1.html

Lets say you want to SSH to a server and run a remote command, ls for instance, with a command like this:

ssh user@server "ls -lh file"

If you are not using SSH keys, you will be prompted a password, and will need to enter it manually.

If you want to avoid entering a password, and have it inputted to login prompt automatically you can use expect command.

Provide SSH password inside a script.

Expect reads cmdfile for a list of commands to execute. Expect may also be invoked implicitly on systems which support the #! notation by marking the script executable, and making the first line in your script:
#!/usr/local/bin/expect -f

Example of a script which runs a remote command over SSH, with password being provided inside a script.

#!/usr/bin/expect -f
spawn ssh [email protected] "ls /file"
expect "assword:"
send "mypassword\r"
interact

One-liner

Running a SSH command with provided password inside a one-liner, can be done using expect -c and then putting the commands inside single quotes.

The -c flag prefaces a command to be executed before any in the script. The command should be quoted to prevent being broken up by the shell. This option may be used multiple times. Multiple commands may be executed with a single -c by separating them with semicolons.

Example of a one-line command:

expect -c 'spawn ssh user@server "ls -lh file"; expect "assword:"; send "mypassword\r"; interact'

Using sshpass

SSH password prompt can also be bypassed by connecting with sshpass command, which is available in EPEL repo in CentOS

To install sshpass, first make sure you have EPEL repo on your server.

You can install EPEL with

yum -y install epel-release

Install sshpass with

yum -y install sshpass

You can then connect to remote server with sshpass using a command similar to this:

sshpass -p 'password' ssh user@server

If you have never connected to the server before, you will probably not get connected to remote server due to host key checking.
To bypass host key checking use -o StrictHostKeyChecking=no option:

sshpass -p 'password' ssh -o StrictHostKeyChecking=no  user@server
References:

http://unix.stackexchange.com/questions/252777/use-she-bang-in-oneliner

http://stackoverflow.com/questions/1924464/bash-controlling-ssh

http://stackoverflow.com/questions/16928004/how-to-enter-ssh-password-using-bash

http://linux.die.net/man/1/expect

http://www.cyberciti.biz/faq/noninteractive-shell-script-ssh-password-provider/

Find jobs for SysAdmins with Jooble

cdp_io processes stuck, causing high load or filling up disk space

R1Soft CDP Agent can sometimes have issues with its backup agent dying, and leaving its cdp_io processes running.

This can cause issues with high load on the server, disk space on the server being filled up by CDP agent, or backup policies on CDP server failing, as they won’t be able to start due to stuck cdp_io processes.

If you do ps auxf | grep cdp you might see a lot of processes like this.

root      587475  0.0  0.3 2136156 128260 ?      Ss   Dec19   0:20 cdp_io/1/0

root      587481  0.0  0.3 2136156 128260 ?      Ss   Dec19   0:14 cdp_io/1/1

root      587482  0.0  0.3 2136156 128260 ?      Ss   Dec19   0:12 cdp_io/1/2

root      587483  0.0  0.3 2136156 128260 ?      Ss   Dec19   0:13 cdp_io/1/3

root      587487  0.0  0.3 2136156 128260 ?      Ss   Dec19   0:13 cdp_io/1/4

root      587488  0.0  0.3 2136156 128260 ?      Ss   Dec19   0:13 cdp_io/1/5

root      587490  0.0  0.3 2136156 128260 ?      Ss   Dec19   0:12 cdp_io/1/6

root      587557  0.0  0.3 2136156 128260 ?      Ss   Dec19   0:12 cdp_io/1/7

root      587603  0.0  0.3 2136156 128260 ?      Ss   Dec19   0:12 cdp_io/1/8

root      587604  0.0  0.3 2136156 128260 ?      Ss   Dec19   0:12 cdp_io/1/9

root      587605  0.0  0.3 2136156 128260 ?      Ss   Dec19   0:12 cdp_io/1/10

root      587607  0.0  0.3 2136156 128260 ?      Ss   Dec19   0:12 cdp_io/1/11

root      587613  0.0  0.3 2136156 128260 ?      Ss   Dec19   0:13 cdp_io/1/12

root      587691  0.0  0.3 2136156 128260 ?      Ss   Dec19   0:13 cdp_io/1/13

root      587699  0.0  0.3 2136156 128260 ?      Ss   Dec19   0:12 cdp_io/1/14

root      587710  0.0  0.3 2136156 128260 ?      Ss   Dec19   0:13 cdp_io/1/15

Trying to kill those processes with kill -9 will yield no results, and processes will be left running.

Doing a Google search on similar issues returns two possible options to kill those processes, and resolve any possible issues caused by them.

http://wiki.r1soft.com/display/CDP/Solved+Issue+of+CDP+Agent+Dying+During+Backup+Process

http://www.webhostingtalk.com/showthread.php?t=1331701

Three possible options to kill the stuck processes are:

  1. Reboot of the device where cdp_io processes are stuck.
  2. Doing a complete remove, and reinstall of CDP agent.
    NOTE: yum reinstall will not work, you need to remove the packages completely, and install them again.
  3. Doing an update of CDP agent to new version, if newer version is available.

On RedHat-compatible distributions, update to a new version can be made by issuing yum update serverbackup*, or in some cases yum update r1soft* command in shell of the device with CDP agent.

Remove and reinstall CDP agent on RedHat-compatible distributions.

To completely remove, and install CDP agent again, first check what CDP agent packages you have installed on your device.

Check currently installed version.

Currently installed CDP agent packages can be checked with rpm -qa | egrep "serverbackup|r1soft", which will show currently installed serverbackup packages, or r1soft packages, depending on which naming version of the packages you have installed on your machine.

[~]# rpm -qa | egrep "serverbackup|r1soft"
serverbackup-enterprise-agent-5.10.1-8.x86_64
serverbackup-async-agent-2-6-5.12.0-21.x86_64
serverbackup-agent-5.12.0-21.x86_64
serverbackup-setup-5.12.0-21.x86_64

Remove and reinstall CDP agent packages.

Remove CDP agent packages with yum remove serverbackup* or yum remove r1soft*, depending on which packages are installed on your machine.

After you remove the packages, all cdp_io processes should be removed, and any load or disk space usage that was caused by the processes will go down.

Reinstall CDP agent packages again with yum install command on packages you had installed before removal.

For example, for the output of rpm -qa | egrep “serverbackup|r1soft” in the example above, you will run the following command.

[~]# yum install serverbackup-agent serverbackup-async-agent-2-6 serverbackup-enterprise-agent serverbackup-setup

..

How to Install ionCube Loader 5 on cPanel/WHM server

At this moment cPanel does not support ionCube Loader 5 on WHM/cPanel servers, which will cause issues for client running files made with ionCube v9.

cPanel currently installs ionCube PHP Loader v4.7.5, when you install PHP using cPanel EasyApache utility.

Your standard php -v output on cPanel server might look something like this.

root@server [~]# php -v
PHP 5.4.45 (cli) (built: Dec 14 2015 17:18:43)
Copyright (c) 1997-2014 The PHP Group
Zend Engine v2.4.0, Copyright (c) 1998-2014 Zend Technologies
with the ionCube PHP Loader v4.7.5, Copyright (c) 2002-2014, by ionCube Ltd.

If you want to install, or upgrade current ionCube PHP Loader on your server to version 5, you can do so manually by downloading the latest version from http://www.ioncube.com/loaders.php, and editing your global or account custom php.ini file.

How to install ionCube Loader manually on cPanel or standard Linux servers with no control panel.

Download the latest loader to your server from http://www.ioncube.com/loaders.php.

If you are running 32 bit server used the following command:

wget http://downloads3.ioncube.com/loader_downloads/ioncube_loaders_lin_x86.tar.gz

If you are running 64 bit server use the following command:

wget http://downloads3.ioncube.com/loader_downloads/ioncube_loaders_lin_x86-64.tar.gz

Unpack the package:

On 32 bit server:

tar xvzf ioncube_loaders_lin_x86.tar.gz

On 64 bit server:

tar xvzf ioncube_loaders_lin_x86-64.tar.gz

It will create ioncube folder in the same directory where you downloaded the file.

If you need to install ionCube Loader for PHP 5.4  you will use ioncube_loader_lin_5.4* files, and if you need to install it for PHP 5.5 you will use ioncube_loader_lin_5.5* files, and similar for other PHP versions.

root@server [~]# ls ioncube
./                             ioncube_loader_lin_4.4_ts.so*  ioncube_loader_lin_5.3.so*     ioncube_loader_lin_5.6_ts.so*
../                            ioncube_loader_lin_5.0.so*     ioncube_loader_lin_5.3_ts.so*  LICENSE.txt
ioncube_loader_lin_4.1.so*     ioncube_loader_lin_5.0_ts.so*  ioncube_loader_lin_5.4.so*     loader-wizard.php
ioncube_loader_lin_4.2.so*     ioncube_loader_lin_5.1.so*     ioncube_loader_lin_5.4_ts.so*  README.txt
ioncube_loader_lin_4.3.so*     ioncube_loader_lin_5.1_ts.so*  ioncube_loader_lin_5.5.so*     USER-GUIDE.md
ioncube_loader_lin_4.3_ts.so*  ioncube_loader_lin_5.2.so*     ioncube_loader_lin_5.5_ts.so*  USER-GUIDE.txt
ioncube_loader_lin_4.4.so*     ioncube_loader_lin_5.2_ts.so*  ioncube_loader_lin_5.6.so*
root@server [~]#

To replace the ionCube Loader for all the users on the server, you can just replace the extension file defined in the global php.ini at /usr/local/lib/php.ini

root@server [~]# grep zend_extension /usr/local/lib/php.ini
zend_extension="/usr/local/IonCube/ioncube_loader_lin_5.4.so"
root@server [~]#

You can rename the original file, for restore purposes, if something goes wrong, and place the new version file from ioncube folder that was created with package extraction.

mv /usr/local/IonCube/ioncube_loader_lin_5.4_ts.so /usr/local/IonCube/ioncube_loader_lin_5.4_ts.so_cporig
mv /usr/local/IonCube/ioncube_loader_lin_5.4.so /usr/local/IonCube/ioncube_loader_lin_5.4.so_cporig
cp ioncube/ioncube_loader_lin_5.4.so /usr/local/IonCube/
cp ioncube/ioncube_loader_lin_5.4_ts.so /usr/local/IonCube/

You can confirm the new version of ionCube Loader with php -v, and look for “ionCube PHP Loader” part of the output.

root@server [~]# php -v
PHP 5.4.45 (cli) (built: Dec 14 2015 17:18:43)
Copyright (c) 1997-2014 The PHP Group
Zend Engine v2.4.0, Copyright (c) 1998-2014 Zend Technologies
    with the ionCube PHP Loader (enabled) + Intrusion Protection from ioncube24.com (unconfigured) v5.0.19, Copyright (c) 2002-2015, by ionCube Ltd.
root@server [~]# php -v | grep "ionCube PHP Loader"
    with the ionCube PHP Loader (enabled) + Intrusion Protection from ioncube24.com (unconfigured) v5.0.19, Copyright (c) 2002-2015, by ionCube Ltd.
root@server [~]#

Upgrade to ionCube Loader 5 on one cPanel account.

If you wanted to change the ion Cube Loader version for only one cPanel account, you can place the ioncube_loader_lin* files for the PHP version used by your client to some custom folder, and define zend_extension values inside a custom php.ini folder on a specific cPanel account.

tl;dr

ionCube Loader can be installed manually on any Linux server with following steps:

  1. Download latest version of ionCube Loaders from http://www.ioncube.com/loaders.php
  2. Unpack the downloaded package
  3. move the ioncube_loader_lin_* files for your PHP version to your extension folder.
  4. Point to the corresponding file in your php.ini file, example
    zend_extension="/usr/local/IonCube/ioncube_loader_lin_5.4.so"

 

MySQL failing to start with message “Can’t find file: ‘./mysql/plugin.frm’ (errno: 23)”

This is a repost of a post from an old blog, made on December 28, 2013, that used to be on:

http://adminramble.com/mysql-failing-start-message-cant-find-file-mysqlplugin-frm-errno-23/

Original post:

Recently I had a problem, where MySQL service was failing to start.
When tailing the MySQL log the following would be recorded while service was being started.

131224 06:04:53 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
/usr/libexec/mysqld: Can't find file: './mysql/plugin.frm' (errno: 23)
131224 6:04:53 [ERROR] Can't open the mysql.plugin table. Please run mysql_upgrade to create it.
/usr/libexec/mysqld: Can't create/write to file '/tmp/ibqcFQMW' (Errcode: 23)
131224 6:04:53 InnoDB: Error: unable to create temporary file; errno: 23
131224 6:04:53 [ERROR] Plugin 'InnoDB' init function returned error.
131224 6:04:53 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
131224 6:04:53 [ERROR] Can't create IP socket: Too many open files in system
131224 6:04:53 [ERROR] Aborting

131224 6:04:53 [Note] /usr/libexec/mysqld: Shutdown complete

131224 06:04:53 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended

Now, the message ‘Can’t find file: ‘./mysql/plugin.frm’ (errno: 23)‘ at the begging of the startup process might make you think that the problems is a missing file, but you can see that at the end of a startup process  this message is logged ‘[ERROR] Can’t create IP socket: Too many open files in system‘.

This suggest that there is a problem with the number of files that is open on the system.
You can confirm this by using a perror utility, which prints system error messages.

If we check the error number 23, which is reported in the error message, we see that the cause of the failure is not a missing file, but that the file can’t be open because to many file handles are in use.

# perror 23
OS error code 23: Too many open files in system

You can check the current maximum number of file descriptors by checking the fs.file-max value in /etc/sysctl.conf, or use the sysctl command to check the current value.

# sysctl -a | grep file-max
 fs.file-max = 65536

To increase the maximum number of file handlesm you can edit /etc/sysctl.conf, change the value of fs.file-max to 200000 or some other value higher then the one you currently have, and then run sysctl -p to apply the new value to the system.

Now, after the file handle number has been increased, you should be able to start the MySQL service normally.