Geoff Manning

I needed to be able to automate the download of specifically named files from an FTP server and then delete them upon successful download. The need arose when the old script I was using was downloading some zero byte files (likely due to poor network conditions, workstation issues or remote server issues) then deleting the source file! D’oh!!

This script checks that the file is not a zero byte file before deleting from the remote server. Not great error/consistency checking, but better than I had before. It also copies the successfully downloaded files to an IMPORT directory and then archives the originally downloaded files to an ARCHIVE directory.

Hopefully you find it useful.

Note: I updated the post to include logging.

#!/bin/bash
################################################
# Set some of the variables that will be needed
# during the execution of this script
################################################
# set current date
now=`date +%Y-%m-%d-%H%M-%S`
#
# set the root dir
rootdir="/ftp_files"
#
# set the download directory
dldir="$rootdir/DOWNLOADS"
#
# set session download dir
sessdldir="$dldir/$now"
#
# set the import dir
importdir="$rootdir/IMPORT"
#
# set the archive dir
archivedir="$rootdir/ARCHIVE"
#
# set up logging
LOGFILE="$rootdir/LOGS/ftplog.txt"
#
################################################
# Lets get to work
#################################################
# create a directory based on todays date
mkdir $sessdldir
#
# change to the new directory prior to FTP connection
cd $sessdldir
#
if [ -f $LOGFILE ]
then
echo "****** FTP PROCESS STARTED AT: $now ******" >> $LOGFILE
else
echo "****** NEW LOG FILE CREATED AT: $now ******" > $LOGFILE
echo "****** FTP PROCESS STARTING AT: $now ******" >> $LOGFILE
fi

## FTP Connection parameters
ftpserver=server.com
username=youruser
password=yourpass
remotedir=/directoryname

## Connect to FTP server and download all TXT files
ftp -inv $ftpserver >> $LOGFILE <<MYEND
user $username $password
cd $remotedir
bin
mget *.TXT
bye
MYEND

## Test for existence of files in the session download dir
## (we cd'd here earlier)
if [ `ls | wc -l` -gt 0 ]
then
echo "" >> $LOGFILE
echo ">> Files have been downloaded" >> $LOGFILE
dlfiles=`ls`
echo ">>>> Downloaded files:" >> $LOGFILE
echo "$dlfiles" >> $LOGFILE
echo "" >> $LOGFILE

## Create the FTP delete script in order to remove the downloaded
## files from the FTP server. We will be sure to check for empty
## files below. The FTP delete script will be created in the session
## download directory and named ftp.$PID
echo "open $ftpserver
user $username $password
binary
cd $remotedir" > $sessdldir/ftp.$$
echo ">> Creating delete script to delete non-zero files from remote server" >> $LOGFILE

## Iterate through the downloaded files one by one
for i in $( ls *.TXT ); do
## If the file exists and is larger than 0 bytes, THEN add
## a delete command to the FTP delete script. This will allow
## us to delete all non-zero byte files from the FTP server.
## Non-zero byte files could be a sign that the file didn't
## download properly due to an FTP timeout or other network
## issues. By not deleting the non-zero byte file we can attempt
## to download it the next time the script is run.
if [ -s $i ]
then
echo "del $i" >> $sessdldir/ftp.$$
echo ">>>> $i will be added to delete script" >> $LOGFILE
fi
done

## Finish off the FTP delete script
echo "quit" >> $sessdldir/ftp.$$

echo "" >> $LOGFILE
echo ">> Here is the delete script">> $LOGFILE
echo "" >> $LOGFILE
cat $sessdldir/ftp.$$ >> $LOGFILE
echo "" >> $LOGFILE
echo ">> Running the delete script" >> $LOGFILE
echo "" >> $LOGFILE

## Run the FTP script
ftp -ivn < $sessdldir/ftp.$$ >> $LOGFILE

## Copy all the non-zero byte TXT files to the import folder
echo "" >> $LOGFILE
echo ">> Copying non-zero size files to IMPORT" >> $LOGFILE

for i in $( ls *.TXT ); do
if [ -s $i ]
then
cp $i $importdir
echo ">>>> Copying $i to IMPORT directory" >> $LOGFILE
fi
done
fi

## Copy session specific download dir to the ARCHIVE folder
echo "" >> $LOGFILE
echo ">> Copying entire $sessdldir to $archivedir" >> $LOGFILE

cp -R $sessdldir $archivedir/

## Delete the session specific download dir
rm -rf $sessdldir
echo "" >> $LOGFILE
echo ">> Deleting $sessdldir" >> $LOGFILE
echo "" >> $LOGFILE

## ENDING LOG FILE FOR SESSION
echo "****** FTP PROCESS ENDED AT: $now ******" >> $LOGFILE
echo "******************************************************" >> $LOGFILE
echo "" >> $LOGFILE

Please consider subscribing to GeoffManning.com to receive new posts in your RSS Reader or by Email.

I have been working with a great nonprofit group in Boston called the Lawyers Clearinghouse on a pro bono basis to redesign their website for the past several months. Along with a friend, Jeff Baker of Partner Alliances, we have redesigned their website and provided them with a way to manage the content using Wordpress.

A little about the Lawyers Clearinghouse:

The Lawyers Clearinghouse provides pro bono legal services to nonprofit organizations, groups seeking nonprofit status, and guests of area homeless shelters. We offer lawyers meaningful and rewarding pro bono opportunities. We also provide educational programs for nonprofit organizations and lawyers.

The project consisted of coming up with new creative for the site, creating a custom Wordpress theme and carrying over the content from the existing site. From there, the group at the Lawyers Clearinghouse made content updates to get it ready for launch.

Here are a few screen captures of the site we started with:

The original site was very informative but dated. They also required a web designer to make simple content changes for them which made it difficult and time consuming to update the site. The goal of the project was to give the Lawyers Clearinghouse a modern look and provide them with the tools to manage the content themselves.

As a nonprofit, it is difficult to justify costs for web design services. It is equally as difficult to make rapid web changes when you need to work around the schedules of designers who are donating their time. So providing a content management system was key to the success of the project.

Here are a few screen shots of the launched site (live site here):

I want to thank Jeff Baker for his help with this project. The creative (with the exception of the logo) is entirely his work. I also want to thank David Bissett (of Dimension Media) for his custom Wordpress theme based off of Jeff’s creative.

People always talk about the power of twitter, it’s ability to effect change (through grassroots fund raising) and mass distribution of info (see California Firefighters) among them. But when I sent out a tweet minutes after my daughter Ella was born I received some quick congratulations from people I have never met face-to-face (but hope to some day).

So Thank You to all who have given congrats and well wishes on what is by far the most exciting time in our lives. And for a quick update, mom and Ella are doing great. We have a pretty good system down so far and have been able to get our sleep in when we can. This is all subject to change from what I have heard

My daughter Ella was born Thursday morning at 5:17 am. She is 6lbs 13oz and 19″ long. She has ten fingers, ten toes and is as cute as can be. I have never been so proud nor held anything so precious in my arms before. It truly is a remarkable feeling. Also, I can’t say it enough how proud I am of my wife.

We are on day 3 at the hospital (Mass General in Boston) and the staff has been incredibly helpful, knowledgeable and kind. We go home this afternoon and that will be an entirely different experience.

Thanks everyone for the support!

It has been way too quiet at GeoffManning.com lately. My wife and I are getting ready for our first child (any day now) and there are some big changes coming up career wise. So both of those have consumed most of my time lately.

What Else I am Working On:

1. I am working with a friend at Partner Alliances on a site redesign for a Boston based Nonprofit Legal Assistance organization. The site is in Wordpress using a custom theme designed by my friend Jeff Baker (of Partner Alliances) and themed by David Bissett at Dimension Media.
2. I am helping out a friend launch her photography website. The site is Wordpress using the Sharpfolio theme created by WebRevolutionary.
3. Preparing for baby!!

I recently had the opportunity to exchange questions via email with Aaron Wall of SEOBook.com on a topic of my choosing in the area of Search Engine Marketing/Search Engine Optimization. My topic was how nonprofits can leverage search engine optimization and marketing tools to further their cause.

[ Geoff ] The recent announcement of the Google Grants program will certainly help level the playing field for non-profits. Could you tell me in your opinion just how important the Google Grants program will be for non-profits from an SEM standpoint?

[ Aaron ] The one area where Google Grants will help bring in direct revenues is for some fake charities set up to push business interests, where people donate valuable goods that get auctioned off in order to receive a tax break. Donate your car offers come to mind, because the tax loophole makes many people donating not care who gets the donation. Some donors may even shop for the maximum write off potential.

But for real charities, I think Google Grants should help them somewhat on an awareness front, but not to directly achieve donations…more to help them bring about awareness of issues that they support. Many non-profits have enough link equity that if they use some of the leading keyword research tools, use clarity in their language, and then make sure they optimize their page titles to focus on issues like world hunger, starving children, etc. that they should be able to get exposure for the issues. Then if people decide they care about those issues and how the non-profit wants to address them then they may donate and/or subscribe.

[ Geoff ] On SEOBook.com you have an article called “101 Link Building Tips to Market Your Website”. Do you feel there are certain link building methods on that list that are more important to non-profits than others methods? Would you revise that list at all for non-profits or do they apply across the spectrum of businesses and industries?

[ Aaron ] I think that even more than with other issues, non-profits tend to be budget limited and fight some of the more destructive elements of corporate externalities.

Given their limited capital (and the vast capital of some competing forces in many markets) non-profits really need to connect to people on an emotional level. They need to make people enraged or concerned, and they need to turn their readers/watchers/learners into teachers to help spread their message.

Here are 6 concepts stemming from that idea

1. Syndicate content on sites like YouTube, like UNICEF did. This does 4 powerful things:
   • Video content is quite captivating, and helps make the invisible visible (see the above UNICEF video)
   • YouTube is a high authority site which has a lot of viewers
   • YouTube foots the bandwidth bill
   • Embeddable videos are easy to spread…and make your message go viral
2. Use newsletters and create a central blog people can subscribe to.
3. Some non-profits may even want to encourage guest bloggers from outsiders who really are passionate about the organization.
4. Outside of videos it helps to have other widgets too…sites like Kiva allow donors to become the marketers by syndicating a widget to their website.
5. Get people to feel involved in the process…let them feel ownership. I recently saw Brave New Films allow people to sponsor a message in the movie. Of course anyone who does that not only helps pay for the film, but they will also buy a copy, and watch it with their friends.
6. If possible try to sell media or other goodies in pairs to try to get people to share it with friends. Host bring a friend nights, etc.

I appreciate and thank Aaron taking the time to answer my questions. You can learn more about Aaron and SEO/SEM at SEOBook.com

Gary Vaynerchuk (from Wine Library TV) has marked today, April 3 2008 as “Good People Day”. The point behind it is to sing the praises of “good people” through your communication vehicle of choice: blog, twitter, pownce, facebook, etc.

Now, this assumes that you are creating media in those channels. If you aren’t, then take the time to talk about or make a recommendation for these “good people”. People that are doing good things in your community whether it is a physical place or an online community.

If you want an example of who I think is “good people” and really helping out communities that I am a part of then please visit these people. If you like what they are saying please let THEM know about it. It’s not an exhaustive list but these people are doing great things:

• CC Chapman
• Christopher S Penn
• Chris Brogan
• Curt Schilling
• Gary Vaynerchuk
• Jeff Pulver
• John Wall
• Mitch Joel

If you were to help bring an old-world marketing person into the new world of social media, how would you do it? What would you tell them? With so many great minds out there sharing their knowledge with all of us, many of what you would need to tell them has already been said.

I would like to work with a few close friends in the old-world of marketing to help bring them forward. Get them into the fishbowl with the rest of us and try to shatter this thing (as CC Chapman would say!).

Here is a list of the marketing blogs and podcasts that I currently follow:

• Awaken your Superhero - Christopher S Penn
• ChrisBrogan.com - Chris Brogan
• Gary Vaynerchuk - Gary Vaynerchuk
• Jaffe Juice - Joseph Jaffe
• Managing the Gray - CC Chapman
• Marketing over Coffee - Christopher S Penn and John Wall
• Ronin Marketeer - John Wall
• Seth’s Blog - Seth Godin
• Six Pixels of Separation - Mitch Joel

And here is my challenge:

To the community: I challenge you to fill in the gaps above. While they are all brilliant, I can’t be smart enough to have compiled a complete list.

To the bloggers and podcasters: What is your one or two best posts or episodes that would really explain the fundamental shift towards social media, networks, etc. that is currently happening?

Have a nagging computer or server issue? DNS or email not working the way you expect? I am offering free technical support/advice tonight March 28th 2008. If you have any questions or issues you can email them to me at geoff@geoffmanning.com and I will try to answer as many questions as I can.

Example topics:

• Desktop/Server issues
• email
• DNS
• Web hosting
• Wordpress
• Spam
• Anti-virus
• etc.

Note: This is not to be confused with the show Chris Pirillo has hosted in the past called: Free Live Tech Support Friday. I will try to answer as many questions as I can but I can’t make any guarantees! If I need to, I can connect remotely via my CrossLoop account.

I received an email from a close friend of mine today that happens to be a runner. He started running marathons last year and has battled through some tough injuries that would have certainly forced me to quit.

This year he wants to run for a charity, a cause. Here is the email he sent me (reprinted with his permission):

As you know I started running marathons last year. I completed the San Diego marathon in 4 hrs 27 minutes, the Cape Cod Marathon in 4 hrs 4 minutes and the Hartford 1/2 Marathon. I logged over 1300 miles and 6 pairs of sneakers last year. I also spent 12 weeks in physical therapy (bad knees).

Now I want to do it all over again this year, except I want to do it to help out other people. This year I am going to run for Curt’s Pitch for ALS (better known as Lou Gehrig’s disease). I will be the first EVER to run the San Diego Marathon for Curt’s Pitch. If you could please help me raise money for this devastating disease, I and the ALS patients would really appreciate it.

If you would like to help out my friend please consider a donation by way of a check made out to:

The ALS Association MA Chapter

The check can be sent to Matt’s home:

Matt Labrie
82 West Central Street
Natick, Ma 01760

Please check out the Curt’s Pitch for ALS website for more information about the charity. The website also has more information about Amyotrophic Lateral Sclerosis (ALS).