My workplace :)

Yep, there's a AT-AT.

Log transfer and update script

The ideas of this script are:
-Download all the files for yesterday (the date must be inside the filename) and save in disk for 61 days.
-Download all the new files generated any day for other days (something like "residual logs... the system generates the log for the day n-1 the n day, but maybe any of the n-m (m<61) day, the system generates another log for any of the n-m days... and the script must download it, verifying the content for every day in the ftp against the disk... it's that clear :P

So, it's not very finished yet, but here is:
(I know, all the comments and vars are in spanish... but just watch a Cheech and Chong's movie and you'll be ok)

AYER=`/bin/date +"%Y%m%d" --date "24 hour ago"`
LOGFILE="/path/to/log"
REMOTEHOST="x.x.x.x"
USER="user"
PASS="pass"
DESTINO="/path/to/disk"

#### ARCHIVOS MENSAJERIA DIA ANTERIOR

if [ ! -d  $DESTINO ]; then
   /bin/mkdir -p $DESTINO
fi

if [ ! -d  $LOGFILE ]; then
   touch $LOGFILE
fi

echo "=Inicio mensajeria============================================" >> $LOGFILE
echo "`/bin/date`" >> $LOGFILE
wget --no-passive-ftp -c -nv -P ${DESTINO} ftp://${USER}:${PASS}@${REMOTEHOST}:/*$AYER* >> $LOGFILE 2>&1
echo "`/bin/date`" >> $LOGFILE
echo "=Fin de mensajeria============================================" >> $LOGFILE

#### TRAYENDO ARCHIVOS REZAGADOS

cat /dev/null > rezagados_m.sh
curl -u$USER:$PASS -l ftp://$REMOTEHOST > ftp.lst
d=1
echo "Buscando rezagados mensajeria============================================" >> $LOGFILE
echo "`/bin/date`" >> $LOGFILE
while [ $d -le 61 ]
do
        old=`/bin/date +"%Y%m%d" --date "$d days ago"`
        grep $old ftp.lst > $old.ftp
        ls ${DESTINO}|grep $old > $old.hdd
        grep -v $old.ftp -f $old.hdd |awk -v destino=${DESTINO} -v user=${USER} -v pass=${PASS} -v remotehost=${REMOTEHOST} '{print "wget --no-passive-ftp -c -nv -P "destino" ftp://"user":"pass"@"remotehost":/"$1}' >> rezagados_m.sh
        if [ `grep -v $old.ftp -f $old.hdd |wc -l` -gt '0'  ]; then
                echo " Se encontraron `grep -v $old.ftp -f $old.hdd |wc -l` archivos rezagados para el dia $old" >> $LOGFILE 2>&1
        fi
        rm $old.*
(( d++ ))
done
echo "`/bin/date`" >> $LOGFILE
echo "Fin busqueda rezagados mensajeria============================================" >> $LOGFILE
echo "Bajando rezagados mensajeria============================================" >> $LOGFILE
bash ./rezagados_m.sh >> $LOGFILE 2>&1
rm -rf ftp.lst
rm -rf rezagados*

I'm not codeconomics  gifted :)

Display execution data inside awk command

The easiest way to use output from, i.e, a command inside awk, is capturing the output and assign it to a variable. For example, if we need to print the current date, we can use:

awk -v D=$(date +%Y%m%d-%H%M) '{print D "-"$12}'

Obviously, you've to customize it according your needs.