The ideas of this script are:
-Download all the files for yesterday (the date must be inside the filename) and save in disk for 61 days.
-Download all the new files generated any day for other days (something like "residual logs... the system generates the log for the day n-1 the n day, but maybe any of the n-m (m<61) day, the system generates another log for any of the n-m days... and the script must download it, verifying the content for every day in the ftp against the disk... it's that clear :P
So, it's not very finished yet, but here is:
(I know, all the comments and vars are in spanish... but just watch a Cheech and Chong's movie and you'll be ok)
AYER=`/bin/date +"%Y%m%d" --date "24 hour ago"`
LOGFILE="/path/to/log"
REMOTEHOST="x.x.x.x"
USER="user"
PASS="pass"
DESTINO="/path/to/disk"
#### ARCHIVOS MENSAJERIA DIA ANTERIOR
if [ ! -d $DESTINO ]; then
/bin/mkdir -p $DESTINO
fi
if [ ! -d $LOGFILE ]; then
touch $LOGFILE
fi
echo "=Inicio mensajeria============================================" >> $LOGFILE
echo "`/bin/date`" >> $LOGFILE
wget --no-passive-ftp -c -nv -P ${DESTINO} ftp://${USER}:${PASS}@${REMOTEHOST}:/*$AYER* >> $LOGFILE 2>&1
echo "`/bin/date`" >> $LOGFILE
echo "=Fin de mensajeria============================================" >> $LOGFILE
#### TRAYENDO ARCHIVOS REZAGADOS
cat /dev/null > rezagados_m.sh
curl -u$USER:$PASS -l ftp://$REMOTEHOST > ftp.lst
d=1
echo "Buscando rezagados mensajeria============================================" >> $LOGFILE
echo "`/bin/date`" >> $LOGFILE
while [ $d -le 61 ]
do
old=`/bin/date +"%Y%m%d" --date "$d days ago"`
grep $old ftp.lst > $old.ftp
ls ${DESTINO}|grep $old > $old.hdd
grep -v $old.ftp -f $old.hdd |awk -v destino=${DESTINO} -v user=${USER} -v pass=${PASS} -v remotehost=${REMOTEHOST} '{print "wget --no-passive-ftp -c -nv -P "destino" ftp://"user":"pass"@"remotehost":/"$1}' >> rezagados_m.sh
if [ `grep -v $old.ftp -f $old.hdd |wc -l` -gt '0' ]; then
echo " Se encontraron `grep -v $old.ftp -f $old.hdd |wc -l` archivos rezagados para el dia $old" >> $LOGFILE 2>&1
fi
rm $old.*
(( d++ ))
done
echo "`/bin/date`" >> $LOGFILE
echo "Fin busqueda rezagados mensajeria============================================" >> $LOGFILE
echo "Bajando rezagados mensajeria============================================" >> $LOGFILE
bash ./rezagados_m.sh >> $LOGFILE 2>&1
rm -rf ftp.lst
rm -rf rezagados*
I'm not codeconomics gifted :)
No comments:
Post a Comment