TODO: Documentation for server database cleanup

Content from an email follows below.

there is a known issue on Linux called the iNode issue - this is when the file system runs out of iNodes, regardless how much space is actually on the drive. you can normally check file space:

$ df

Filesystem 1K-blocks Used Available Use% Mounted on

udev 1988912 0 1988912 0% /dev

tmpfs 401352 560 400792 1% /run

/dev/sda1 261967700 162205100 86429352 66% /

tmpfs 2006756 0 2006756 0% /dev/shm

tmpfs 5120 0 5120 0% /run/lock

/dev/sda3 263104580 27601852 222064572 12% /share

tmpfs 401348 0 401348 0% /run/user/1000

but, it is misleading - as you need to check iNodes

$ df -i

Filesystem Inodes IUsed IFree IUse% Mounted on

udev 497228 364 496864 1% /dev

tmpfs 501689 552 501137 1% /run

/dev/sda1 16646144 50840 16595304 1% /

tmpfs 501689 1 501688 1% /dev/shm

tmpfs 501689 2 501687 1% /run/lock

/dev/sda3 16809984 16809984 0 100% /share

tmpfs 100337 17 100320 1% /run/user/1000

aha.. notice the iUse% column.. drive has used all the iNodes.. this means no file can be created.. to fix it.. you need to remove files :) our database uses a LOT of small files.. each communication is recorded as a single file.. so; each device can do 20 per minute, 120 per hour.. 1440 per day.. you get the point :P

it has happened before - the solution was to manually remove files. (basically, delete data from last month)

i have now created a script that archives each day up into a single file..

$ cat sas-archive_data.sh

#!/bin/bash

working directory

cd /share/oasis-data/sas/data.store/tenant/1/device

we need to know yesterdays date in yyyy.mm.dd format

DAY=$(date -d “yesterday 23:00” ‘+%Y.%m.%d’)

process each directory (each device effectively)

for d in [123456789]*; do

[ -L "${d%/}" ] && continue



# create an archive of the data

echo tar czf $d"/"$DAY.tgz $d"/"$DAY

time tar czf $d"/"$DAY.tgz $d"/"$DAY



# remove the old data, we now have it as a .tgz

echo rm -rf $d"/"$DAY

time rm -rf $d"/"$DAY

done

lets see how much space we now have

df -i

and a cronjob entry:

Example of job definition:

.—————- minute (0 - 59)

| .————- hour (0 - 23)

| | .———- day of month (1 - 31)

| | | .——- month (1 - 12) OR jan,feb,mar,apr …

| | | | .—- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat

| | | | |

* * * * * user-name command to be executed

0 3 * * * www-data (cd / && /home/riotsecure/sas-archive_data.sh > /tmp/sas.cronjob.log )

this will run every day in the background at 3am in the morning :) which means we should never run into this problem ever again. can check tomorrow to see if it actually worked by checking the log file that is created in the /tmp directory

Mikael: this should probably be documented as part of the system integration documentation; especially if we are providing the simple script that stored each communication into a single file.