Hello, my name is Matteo Mattei and this is my personal website. I am computer engineer with a long experience in Linux system administration and web software development.
26 May 2016 
Matteo Mattei 
python http https 
The following script performs GET and POST requests to [httpbin.org] using only builtin python 3 modules. There is also a class to support file encoding for upload.
15 Apr 2016 Matteo Mattei arm wget compile The following script can be used to statically cross compile wget for ARM.
Requirements:
--enable-static-nss flag so that getaddrinfo and gethostbyname cannot complain at link time.The resulting binary will be placed into the build folder.
22 Feb 2016 Matteo Mattei server networking firewall shorewall dnsmasq I have a router with a public static ip address provided by the ISP and I need to share internet access to all pc in the LAN. To do it I need a server with two ethernet interfaces (eth0 and eth1) that will act as a firewall and dhcp server. That server will also be used as a web server to publish some contents in the LAN and in internet.
My configuration is this:
Install shorewall:
apt-get install shorewall
Start configuration with two interface shorewall example:
cd /usr/share/doc/shorewall/examples/two-interfaces/
cp interfaces /etc/shorewall/
cp masq /etc/shorewall/
cp policy /etc/shorewall/
cp rules /etc/shorewall/
cp zones /etc/shorewall/
cp stoppedrules /etc/shorewall/
Now configure /etc/network/interfaces (I am using Debian jessie):
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet static
address xxx.xxx.xxx.xxx
netmask yyy.yyy.yyy.yyy
gateway zzz.zzz.zzz.zzz
auto eth1
iface eth1 inet static
address 192.168.0.1
netmask 255.255.255.0
where:
And /etc/resolv.conf:
# Google DNS
nameserver 8.8.8.8
nameserver 8.8.4.4
Now configure shorewall to act as firewall and share internet to all LAN devices.
File: /etc/shorewall/interfaces
#ZONE INTERFACE OPTIONS
net eth0 dhcp,tcpflags,nosmurfs,routefilter,logmartians,sourceroute=0
loc eth1 dhcp,tcpflags,nosmurfs,routefilter,logmartians
File: /etc/shorewall/zones
#ZONE TYPE OPTIONS IN OUT
# OPTIONS OPTIONS
fw firewall
net ipv4
loc ipv4
File: /etc/shorewall/policy
#SOURCE DEST POLICY LOG LEVEL LIMIT:BURST
$FW net ACCEPT
$FW loc ACCEPT
loc net ACCEPT
net all DROP info
# THE FOLLOWING POLICY MUST BE LAST
all all REJECT info
File: /etc/shorewall/rules
?SECTION ALL
?SECTION ESTABLISHED
?SECTION RELATED
?SECTION INVALID
?SECTION UNTRACKED
?SECTION NEW
# Don't allow connection pickup from the net
#
Invalid(DROP) net all tcp
#
# Accept DNS connections from the firewall to the network
#
DNS(ACCEPT) $FW net
DNS(ACCEPT) loc $FW
#
# Accept SSH connections from the local network for administration
#
SSH(ACCEPT) loc $FW
#
# Allow Ping from the local network
#
Ping(ACCEPT) loc $FW
#
# Drop Ping from the "bad" net zone.. and prevent your log from being flooded..
#
Ping(DROP) net $FW
ACCEPT $FW loc icmp
ACCEPT $FW net icmp
#
# custom rules
SSH(ACCEPT) net $FW
Web(ACCEPT) net $FW
Web(ACCEPT) loc $FW
# DNAT rules (useful for natting a service on a device)
# this rule opens port 8080 from internet to port 80 of 192.168.0.2 in TCP
#DNAT net loc:192.168.0.2:80 tcp 8080
The above configuration allows SSH connections from local and from remote as well as Web access.
File: /etc/shorewall/masq
#INTERFACE:DEST SOURCE ADDRESS PROTO PORT(S) IPSEC MARK USER/ SWITCH ORIGINAL
# GROUP DEST
eth0 eth1
File: /etc/shorewall/stoppedrules
#ACTION SOURCE DEST PROTO DEST SOURCE
# PORT(S) PORT(S)
ACCEPT eth1 -
ACCEPT - eth1
Now edit /etc/shorewall/shorewall.conf and set:
STARTUP_ENABLED=Yes
IP_FORWARDING=On
In particular the last line is necessary to share internet to the LAN. Edit now /etc/default/shorewall and set:
startup=1
Now start shorewall:
/etc/init.d/shorewall start
And try to configure a pc in the LAN with a static ip address, for example:
address: 192.168.0.200
netmask: 255.255.255.0
gateway: 192.168.0.1
Yeah, the pc should be able to access internet! But we need to go a little bit ahead because given we don’t want to assign a static ip address to all pc (or devices) in the LAN. So we have to install a DHCP server.
apt-get install dnsmasq
Now backup the default configuration of dnsmasq and create a new /etc/dnsmasq.conf with something like this:
interface=eth1
dhcp-range=192.168.0.50,192.168.0.150,12h
dhcp-host=60:E3:27:8D:88:64,accesspoint,192.168.0.10
dhcp-host=F8:32:E5:84:3A:1F,pc1,192.168.0.11
dhcp-host=F8:32:E5:84:2D:71,pc2,192.168.0.12
dhcp-host=F8:32:E5:84:2D:B6,pc3,192.168.0.13
The first line specifies the interface where the DHCP server is running (eth1 for me1). The second line (dhcp-range) is the range of IP addresses that the DHCP server will provide with a lease of 12 hours. In this case from address 192.168.0.50 to address 192.168.0.150. All other lines are used to define devices with static IPs. The syntax is:
dhcp-host=DEVICE_MAC_ADDRESS,DEVICE_NAME,DEVICE_IP
Dnsmasq beyond dhcp and DNS caching provides also another interesting feature… every entry you set in /etc/hosts of the server is automatically forwarded to all devices in the LAN. This means that if we want to set a simple name for accessing the webserver from the LAN it’s just a matter of editing /etc/hosts and add the server’s name:
127.0.0.1 localhost
192.168.0.1 myserver
This allows all devices in the LAN to access the server using myserver.
Now restart dnsmasq:
/etc/init.d/dnsmasq restart
To list all devices that have received a new IP address the file to look at is /var/lib/misc/dnsmasq.leases
20 Feb 2016 Matteo Mattei linux server mysql varnish debian php iptables postfix cp /etc/skel/.bashrc /root/.bashrc
apt-get update
apt-get dist-upgrade
Make sure to have the following two lines (with the same format) at the top of your /etc/hosts file
127.0.0.1 localhost.localdomain localhost
xxx.xxx.xxx.xxx web1.myserver.com web1
Note: xxx.xxx.xxx.xxx is the public IP address assigned to your server.
apt-get install php5 mysql-server mysql-client apache2 iptables phpmyadmin varnish shorewall vsftpd php5-cli php5-curl php5-dev php5-gd php5-imagick php5-imap php5-memcache php5-pspell php5-recode php5-sqlite php5-tidy php5-xcache php5-xmlrpc php-pear php-xml-rpc postfix apg ca-certificates heirloom-mailx
MySQL/PhpMyAdmin:
Postfix:
Stop VSFTP server:
/etc/init.d/vsftpd stop
Create backup configuration:
mv /etc/vsftpd.conf /etc/vsftpd.conf.backup
Add new configuration:
listen=YES
listen_port=21
anonymous_enable=NO
local_enable=YES
guest_enable=YES
guest_username=nobody
user_sub_token=$USER
local_root=/var/www/vhosts/$USER
virtual_use_local_privs=YES
user_config_dir=/etc/vsftpd/users
pam_service_name=vsftpd_local_and_virtual
chroot_local_user=YES
chroot_list_enable=YES
chroot_list_file=/etc/vsftpd/chroot_list
ftpd_banner=Welcome to my ftp server
write_enable=YES
download_enable=YES
dirlist_enable=YES
local_umask=022
dirmessage_enable=YES
xferlog_enable=YES
xferlog_file=/var/log/xferlog
connect_from_port_20=YES
connect_timeout=60
data_connection_timeout=300
idle_session_timeout=300
local_max_rate=0
max_clients=0
max_per_ip=3
Create an empty chroot_list file:
mkdir /etc/vsftpd
touch /etc/vsftpd/chroot_list
Install PAM module for virtual users:
apt-get install libpam-pwdfile
And configure it creating the file /etc/pam.d/vsftpd_local_and_virtual with this content:
# Standard behaviour for ftpd(8).
auth required pam_listfile.so item=user sense=deny file=/etc/ftpusers onerr=succeed
# first try to authenticate local users
auth sufficient pam_unix.so
# if that failed, login with virtual user
auth required pam_pwdfile.so pwdfile /etc/vsftpd/passwd
# pam_pwdfile doesn't come with account, so we just permit on success
account required pam_permit.so
Start VSFTP server:
/etc/init.d/vsftpd start
Stop Apache web server:
/etc/init.d/apache2 stop
Backup Apache configuration:
cp /etc/apache2/apache2.conf /etc/apache2/apache2.conf.backup
Edit the following lines in /etc/apache2/apache2.conf
Edit /etc/apache2/mods-enabled/mpm_prefork.conf:
<IfModule mpm_prefork_module>
StartServers 5
MinSpareServers 5
MaxSpareServers 10
MaxRequestWorkers 150
MaxConnectionsPerChild 10000
</IfModule>
Edit /etc/apache2/ports.conf and change the port 80 with 8080 since we are going to use Varnish:
Listen 8080
Change the port (from 80 to 8080) also in the default virtual host /etc/apache2/sites-enabled/000-default.conf Now restart Apache:
/etc/init.d/apache2 restart
Stop Varnish daemon:
/etc/init.d/varnish stop
Open /etc/varnish/default.vcl and make sure the backend section is like this:
backend default {
.host = "127.0.0.1";
.port = "8080";
.connect_timeout = 600s;
.first_byte_timeout = 600s;
.between_bytes_timeout = 600s;
}
Now edit /etc/default/varnish and set the DAEMON_OPTS variable like this:
DAEMON_OPTS="-a :80 \
-T localhost:6082 \
-f /etc/varnish/default.vcl \
-S /etc/varnish/secret \
-s malloc,256m"
Now we have to make some changes also to systemd scripts (this step is mandatory for Debian Jessie!) since systemd does not consider /etc/default/varnish settings:
cp /lib/systemd/system/varnish.service /etc/systemd/system/
Edit /etc/systemd/system/varnish.service and change port 6081 with port 80:
[Unit]
Description=Varnish HTTP accelerator
[Service]
Type=forking
LimitNOFILE=131072
LimitMEMLOCK=82000
ExecStartPre=/usr/sbin/varnishd -C -f /etc/varnish/default.vcl
ExecStart=/usr/sbin/varnishd -a :80 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,256m
ExecReload=/usr/share/varnish/reload-vcl
[Install]
WantedBy=multi-user.target
Restart Varnish:
systemctl daemon-reload
systemctl restart varnish.service
Correct the MySQL configuration warning:
sed -i "{s/^key_buffer/key_buffer_size/g}" /etc/mysql/my.cnf
Enable MySQL slow query logging (often useful during slow page load debugging):
sed -i "{s/^#slow_query_log_file /slow_query_log_file /g}" /etc/mysql/my.cnf
sed -i "{s/^#slow_query_log /slow_query_log /g}" /etc/mysql/my.cnf
sed -i "{s/^#long_query_time /long_query_time /g}" /etc/mysql/my.cnf
sed -i "{s/^#log_queries_not_using_indexes/log_queries_not_using_indexes/g}" /etc/mysql/my.cnf
MySQL is now configured, so restart it:
/etc/init.d/mysql restart
Copy the default configuration for one interface:
cd /usr/share/doc/shorewall/examples/one-interface
cp interfaces /etc/shorewall/
cp policy /etc/shorewall/
cp rules /etc/shorewall/
cp zones /etc/shorewall/
Now open /etc/shorewall/policy file and change the line:
net all DROP info
removing info directive given it fills the system logs:
net all DROP
Now open /etc/shorewall/rules and add the following rules at the bottom of the file:
HTTP/ACCEPT net $FW
SSH/ACCEPT net $FW
FTP/ACCEPT net $FW
# real apache since varnish listens on port 80
#ACCEPT net $FW tcp 8080
NOTE: in case you want to allow ICMP (Ping) traffic from a specific remote hosts you need to add a rule similar to the following where xxx.xxx.xxx.xxx is the remote IP address, before the Ping(DROP) rule:
Ping(ACCEPT) net:xxx.xxx.xxx.xxx $FW
Now edit /etc/default/shorewall and change startup=0 to startup=1 You are now ready to start the firewall:
/etc/init.d/shorewall start
Stop postfix server:
/etc/init.d/postfix stop
Edit /etc/mailname and set your server domain name, for example:
server1.mycompany.com
Then, in order to monitor mail traffic coming from PHP you need to edit /etc/php5/apache2/php.ini. Go to [mail function] section and set the following two options:
sendmail_path = /usr/local/bin/sendmail-wrapper
auto_prepend_file = /usr/local/bin/env.php
Now create the two files above in /usr/local/bin:
sendmail-wrapper:
#!/bin/sh
logger -p mail.info sendmail-wrapper.sh: site=${HTTP_HOST}, client=${REMOTE_ADDR}, script=${SCRIPT_NAME}, pwd=${PWD}, uid=${UID}, user=$(whoami)
/usr/sbin/sendmail -t -i $*
env.php:
<?php
putenv("HTTP_HOST=".@$_SERVER["HTTP_HOST"]);
putenv("SCRIPT_NAME=".@$_SERVER["SCRIPT_NAME"]);
putenv("SCRIPT_FILENAME=".@$_SERVER["SCRIPT_FILENAME"]);
putenv("DOCUMENT_ROOT=".@$_SERVER["DOCUMENT_ROOT"]);
putenv("REMOTE_ADDR=".@$_SERVER["REMOTE_ADDR"]);
?>
Now make they both have executable flag:
chmod +x /usr/local/bin/sendmail-wrapper
chmod +x /usr/local/bin/env.php
Add also /usr/local/bin/ to the open_basedir php list in /etc/apache2/conf-enabled/phpmyadmin.conf
php_admin_value open_basedir /usr/share/phpmyadmin/:/etc/phpmyadmin/:/var/lib/phpmyadmin/:/usr/local/bin/
Restart Postfix:
/etc/init.d/postfix start
Create all needed directories and files
mkdir /root/cron_scripts
mkdir -p /var/www/vhosts
mkdir -p /etc/vsftpd/users
touch /etc/vsftpd/passwd
Now download all tools to manage the server locally:
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/Debian7/LAMP_24/ADD_ALIAS.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/Debian7/LAMP_24/ADD_DOMAIN.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/Debian7/LAMP_24/ADD_FTP_VIRTUAL_USER.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/Debian7/LAMP_24/ALIAS_LIST.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/Debian7/LAMP_24/DEL_ALIAS.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/Debian7/LAMP_24/DEL_DOMAIN.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/Debian7/LAMP_24/DEL_FTP_VIRTUAL_USER.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/Debian7/LAMP_24/DOMAIN_LIST.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/Debian7/LAMP_24/MYSQL_CREATE.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/Debian7/LAMP_24/UPDATE_ALL_FTP_PASSWORD.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/Debian7/LAMP_24/UPDATE_FTP_PASSWORD.sh
chmod 770 *.sh
Download also the tools that will be used with cron:
cd /root/cron_scripts
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/Debian7/LAMP_24/cron_scripts/backup_mysql.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/Debian7/LAMP_24/cron_scripts/mysql_optimize.sh
chmod 770 *.sh
Edit /etc/crontab and add the following lines at the bottom:
# mysql optimize tables
3 4 * * 7 root /root/mysql_optimize.sh
# mysql backup
32 4 * * * root /root/backup_mysql.sh
Download latest Raspbian Lite distribution (Jessie in my case) from the official Raspberry website.
Then unzip the file and flash the image in the SD card:
sudo dd if=2016-02-09-raspbian-jessie-lite.img of=/dev/mmcblk0 bs=1M
Power up the RPi2 (or RPi1) with the SD plugged in and log in:
user: pi password: raspberry
Now configure the network (wired static ip in my case) editing /etc/dhcpcd.conf and adding the following lines at the bottom:
# my custom static settings
interface eth0
static ip_address=192.168.1.130/24
static routers=192.168.1.254
static domain_name_servers=192.168.1.254
Execute raspi-config to cutomize some settings:
sudo raspi-config
In particular,
At the end reboot your raspberry pi so that the new filesystem size will take effect.
Login again, update the system and install all needed software:
sudo apt-get update
sudo apt-get dist-upgrade
sudo apt-get install midori matchbox-window-manager xserver-xorg x11-xserver-utils unclutter xinit
Now add tty group to pi user because pi needs to handle /dev/ttyX devices and adjust permissions accordingly at every system startup:
gpasswd -a pi tty
sed -i '/^exit 0/c\chmod g+rw /dev/tty?\nexit 0' /etc/rc.local
Now create a startup script:
vi /home/pi/startkiosk.sh
#!/bin/bash
# disable DPMS (Energy Star) features.
xset -dpms
# disable screen saver
xset s off
# don't blank the video device
xset s noblank
# disable mouse pointer
unclutter &
# run window manager
matchbox-window-manager -use_cursor no -use_titlebar no &
# run browser
midori -e Fullscreen -a http://www.google.com
and make it executable:
chmod +x /home/pi/startkiosk.sh
Then add the following lines at the end of /home/pi/.bashrc:
if [ -z "${SSH_TTY}" ]; then
xinit ~/startkiosk.sh
fi
This three lines allow starting X only when we login as pi directly but not from SSH.
Backup your current /etc/fstab and create a new one with the following content:
proc /proc proc defaults 0 0
/dev/mmcblk0p1 /boot vfat ro 0 2
/dev/mmcblk0p2 / ext4 ro 0 1
tmpfs /tmp tmpfs defaults,noatime,mode=1777 0 0
tmpfs /var/log tmpfs defaults,noatime,mode=0755 0 0
tmpfs /var/lib/systemd tmpfs defaults,noatime,mode=0755 0 0
tmpfs /run tmpfs defaults,noatime,mode=0755 0 0
Then create a little script to help you to change the read-write/read-only mode of the filesystem everytime you need:
vi /home/pi/mountfs.sh
#!/bin/bash
case "${1}" in
rw)
sudo mount -o remount,rw /
echo "Filesystem mounted in READ-WRITE mode"
;;
ro)
sudo mount -o remount,ro /
echo "Filesystem mounted in READ-ONLY mode"
;;
*)
if [ -n "$(mount | grep mmcblk0p2 | grep -o 'rw')" ]
then
echo "Filesystem is mounted in READ-WRITE mode"
else
echo "Filesystem is mounted in READ-ONLY mode"
fi
echo "Usage ${0} [rw|ro]"
;;
esac
That’s it, reboot your RPi and enjoy your safe web kiosk!
09 Dec 2015 Matteo Mattei python This is a very quick guide on how to create self contained Python executables (for all platforms). First of all install PyInstaller (I am using pip3 because I work with Python 3.x):
sudo pip3 install pyinstaller
Now install upx for a better compression:
sudo apt-get install upx
Now you are ready to create your self-contained executable:
pyinstaller \
--onefile \
--noconfirm \
--noconsole \
--clean \
--log-level=WARN \
--key=MySuperSecretPassword \
--strip \
myscript.py
Resulting executable will be placed inside dist folder and it will be called myscript. This is what each parameter does:
--onefile allows to create a single self contained binary.--noconfirm replaces output directory without asking for confirmation.--noconsole should be used in GUI application with no console. --console should be used otherwise.--clean cleans PyInstaller cache and remove temporary files.--log-level=WARN shows only warnings and errors during build.--key=yourkey uses the given key to encrypt the Python bytecode (yes it’s secure!).--strip removes debug information to executable and shared libraries.For windows, if you want to add also the icon to the resulting exe file you can add this additional parameter:
--icon-file=myapplication.ico
The application have only to be recompiled on every platform you want to release your application for. I know, the resulting binary will be a little heavy (~24MB for a PySide GUI application on Linux) but we have to consider that it contains the interpreter itself and all the needed libraries!!!