This could be useful when you really need to shutdown... some kind of a "Vulcan Key" to the kernel :)
#echo 1 > /proc/sys/kernel/sysrq
#echo o > /proc/sysrq-trigger
Showing posts with label bash. Show all posts
Showing posts with label bash. Show all posts
Thursday, 25 April 2013
Thursday, 30 December 2010
Log transfer and update script
The ideas of this script are:
-Download all the files for yesterday (the date must be inside the filename) and save in disk for 61 days.
-Download all the new files generated any day for other days (something like "residual logs... the system generates the log for the day n-1 the n day, but maybe any of the n-m (m<61) day, the system generates another log for any of the n-m days... and the script must download it, verifying the content for every day in the ftp against the disk... it's that clear :P
So, it's not very finished yet, but here is:
(I know, all the comments and vars are in spanish... but just watch a Cheech and Chong's movie and you'll be ok)
AYER=`/bin/date +"%Y%m%d" --date "24 hour ago"`
LOGFILE="/path/to/log"
REMOTEHOST="x.x.x.x"
USER="user"
PASS="pass"
DESTINO="/path/to/disk"
#### ARCHIVOS MENSAJERIA DIA ANTERIOR
if [ ! -d $DESTINO ]; then
/bin/mkdir -p $DESTINO
fi
if [ ! -d $LOGFILE ]; then
touch $LOGFILE
fi
echo "=Inicio mensajeria============================================" >> $LOGFILE
echo "`/bin/date`" >> $LOGFILE
wget --no-passive-ftp -c -nv -P ${DESTINO} ftp://${USER}:${PASS}@${REMOTEHOST}:/*$AYER* >> $LOGFILE 2>&1
echo "`/bin/date`" >> $LOGFILE
echo "=Fin de mensajeria============================================" >> $LOGFILE
#### TRAYENDO ARCHIVOS REZAGADOS
cat /dev/null > rezagados_m.sh
curl -u$USER:$PASS -l ftp://$REMOTEHOST > ftp.lst
d=1
echo "Buscando rezagados mensajeria============================================" >> $LOGFILE
echo "`/bin/date`" >> $LOGFILE
while [ $d -le 61 ]
do
old=`/bin/date +"%Y%m%d" --date "$d days ago"`
grep $old ftp.lst > $old.ftp
ls ${DESTINO}|grep $old > $old.hdd
grep -v $old.ftp -f $old.hdd |awk -v destino=${DESTINO} -v user=${USER} -v pass=${PASS} -v remotehost=${REMOTEHOST} '{print "wget --no-passive-ftp -c -nv -P "destino" ftp://"user":"pass"@"remotehost":/"$1}' >> rezagados_m.sh
if [ `grep -v $old.ftp -f $old.hdd |wc -l` -gt '0' ]; then
echo " Se encontraron `grep -v $old.ftp -f $old.hdd |wc -l` archivos rezagados para el dia $old" >> $LOGFILE 2>&1
fi
rm $old.*
(( d++ ))
done
echo "`/bin/date`" >> $LOGFILE
echo "Fin busqueda rezagados mensajeria============================================" >> $LOGFILE
echo "Bajando rezagados mensajeria============================================" >> $LOGFILE
bash ./rezagados_m.sh >> $LOGFILE 2>&1
rm -rf ftp.lst
rm -rf rezagados*
I'm not codeconomics gifted :)
-Download all the files for yesterday (the date must be inside the filename) and save in disk for 61 days.
-Download all the new files generated any day for other days (something like "residual logs... the system generates the log for the day n-1 the n day, but maybe any of the n-m (m<61) day, the system generates another log for any of the n-m days... and the script must download it, verifying the content for every day in the ftp against the disk... it's that clear :P
So, it's not very finished yet, but here is:
(I know, all the comments and vars are in spanish... but just watch a Cheech and Chong's movie and you'll be ok)
AYER=`/bin/date +"%Y%m%d" --date "24 hour ago"`
LOGFILE="/path/to/log"
REMOTEHOST="x.x.x.x"
USER="user"
PASS="pass"
DESTINO="/path/to/disk"
#### ARCHIVOS MENSAJERIA DIA ANTERIOR
if [ ! -d $DESTINO ]; then
/bin/mkdir -p $DESTINO
fi
if [ ! -d $LOGFILE ]; then
touch $LOGFILE
fi
echo "=Inicio mensajeria============================================" >> $LOGFILE
echo "`/bin/date`" >> $LOGFILE
wget --no-passive-ftp -c -nv -P ${DESTINO} ftp://${USER}:${PASS}@${REMOTEHOST}:/*$AYER* >> $LOGFILE 2>&1
echo "`/bin/date`" >> $LOGFILE
echo "=Fin de mensajeria============================================" >> $LOGFILE
#### TRAYENDO ARCHIVOS REZAGADOS
cat /dev/null > rezagados_m.sh
curl -u$USER:$PASS -l ftp://$REMOTEHOST > ftp.lst
d=1
echo "Buscando rezagados mensajeria============================================" >> $LOGFILE
echo "`/bin/date`" >> $LOGFILE
while [ $d -le 61 ]
do
old=`/bin/date +"%Y%m%d" --date "$d days ago"`
grep $old ftp.lst > $old.ftp
ls ${DESTINO}|grep $old > $old.hdd
grep -v $old.ftp -f $old.hdd |awk -v destino=${DESTINO} -v user=${USER} -v pass=${PASS} -v remotehost=${REMOTEHOST} '{print "wget --no-passive-ftp -c -nv -P "destino" ftp://"user":"pass"@"remotehost":/"$1}' >> rezagados_m.sh
if [ `grep -v $old.ftp -f $old.hdd |wc -l` -gt '0' ]; then
echo " Se encontraron `grep -v $old.ftp -f $old.hdd |wc -l` archivos rezagados para el dia $old" >> $LOGFILE 2>&1
fi
rm $old.*
(( d++ ))
done
echo "`/bin/date`" >> $LOGFILE
echo "Fin busqueda rezagados mensajeria============================================" >> $LOGFILE
echo "Bajando rezagados mensajeria============================================" >> $LOGFILE
bash ./rezagados_m.sh >> $LOGFILE 2>&1
rm -rf ftp.lst
rm -rf rezagados*
I'm not codeconomics gifted :)
Subscribe to:
Posts (Atom)