Saturday, February 11, 2012

How to automatically synchronize a picture folder with picasa (on a NAS or anywhere else)






I like to have a copy of my pictures on picasa to be able to share them with friends and family. I usually upload them in reduced resolution, to stay within the free storage space given by google.
The problem is that uploading them can be a pain: the picasa software can be very slow to upload them, especially if your are accessing the pictures on your NAS with a wireless network.

Instead of using the picasa software, I tried to use googlecl tools (http://code.google.com/p/googlecl/) to do that but it turns out I couldn't get it to do want I want (no sync folder option + no resize of picture on the fly).
There is an unsupported patch to synchronize folders with googlecl (http://code.google.com/p/googlecl/issues/detail?id=170) but that doesn't solve the problem of image resizing...I did not even test it...


1. Presentation of my solution: picasauploader.jar

To solve the problem, I wrote a small piece of java code (picasauploader.jar) that:
- creates any new album in picasa web when a new folder is created on the disk
- upload (and resizes if necessary) new pictures on the disk to picasa web

In my setup, I want to install picasauploader.jar as a daily cron on my NAS, but you can install it anywhere.

You just need to organize your pictures as
/path/albumname/picture.jpg
and use /path in the picasauploader.properties

The jar is configured using the config file picasauploader.properties which reads as follows:
#use system defined proxy
picasauploader.usesystemproxy=true
#picasa/google account username and password
picasauploader.username=xxxx
picasauploader.password=xxx
#semi column separated directories
picasauploader.diskpaths=/xxx/yyyy;/aaaa/bbbb
#can be either:
# private: accessible to anybody with the direct link but not without the direct link
# protected: not accessible except from your account
# public: available for everybody to see
picasauploader.albumcreationaccess=private
#if you want to resize images before uploading (aspect ratio is kept)
#Note: only JPEG images are resized...
#max Height in px
picasauploader.maxheigt=1600
#max Width in px
picasauploader.maxwidth=1600
#jpg quality when resizing
picasauploader.resizequality=85
#log file (for linux, good practice is to put it in /var/log/ or /opt/var/log (and make sur logrotate works correctly))
picasauploader.logfile=/opt/var/log/picasaupload.log

All options are self explanatory. You can customize it as required by your setup.

As the program is java, it can be run on any OS / Architecture supporting Java.

The jar is available for download at http://dl.dropbox.com/u/50398581/picasauploader/picasauploader.jar
sample properties files is available at http://dl.dropbox.com/u/50398581/picasauploader/picasauploader.properties
and source code is available at: http://dl.dropbox.com/u/50398581/picasauploader/PicasaUploader.java

Please note that for safety, the program does not delete anything on picasa web (nor on the disk, of course). Therefore, it is very safe to use.

Known Limitations:
- only suports JPG GIF PNG BMP image formats
- picture resizing is only supported for jpg images
- only the name is used to determine if a picture was already uploaded: if a picture was already uploaded and then changed on disk, it won't be uploaded again.


2. Steps to install the picasauploader on a linux based NAS
The setup is easy to adapt to any machine running linux. I didn't do a tutorial for Windows or Mac as I lack some knowledge to do it, but it can of course be done... feel free to adapt it and post your results and hints in the comments!
This tutorial assumes some vi ans linux knowledge...

This is how I installed the picasauploader.jar on my NAS (an Iomega Storcenter ix4-200d). Please note that the procedure is unsupported by Iomega! use at your own risk!

a. Download and setup of picasauploader
First, you need to ssh into your NAS (see my other post if you have am Iomega Storcenter)
Then:
mkdir /opt/usr/local
mkdir /opt/usr/local/picasauploader
cd /opt/usr/local/picasauploader
wget http://dl.dropbox.com/u/50398581/picasauploader/picasauploader.jar
wget http://dl.dropbox.com/u/50398581/picasauploader/picasauploader.properties
Don't forget to change the properties file to make it work for your setup (you at least need to change account information and paths):
vi picasauploader.properties

If you haven't already done so, you need to install java on your NAS. See the java section of my previous post How to install Crashplan on an Iomega Storcenter to find out how to do it for an Iomega storcenter.

If you followed the java installation procedure of my other post, link java to a more usual location:
ln -s /mnt/pools/A/A0/NAS_Extension/ejre1.7.0/bin/java /opt/bin/java
The setup can already be tested by starting the command:
/opt/bin/java -jar /opt/usr/local/picasauploader/picasauploader.jar /opt/usr/local/picasauploader/picasauploader.properties

b. Set up a cron job to synchronize image folders with picasa
Create the picasauploader cron:
cd /etc/cron.daily/
vi picasauploader
and add:
#!/bin/sh
/opt/bin/java -jar /opt/usr/local/picasauploader/picasauploader.jar /opt/usr/local/picasauploader/picasauploader.properties
Then:
chmod a+x picasauploader
And test with:
./picasauploader

c. start the cron daemon

The cron daemon is not started at boot by default....

You can start it manually:
/etc/init.d/cron start

But to have it start up every time at boot, we need to add the line:
/etc/init.d/cron start >> /opt/init-opt.log
to our /opt/init-opt.sh script.

See my other post How to run a program at boot on the Iomega Storcenter NAS to see how it works!

d. set up logrotate
Logrotate is the process that compresses and delete old logs so that your logs don't eat all you disk space!
vi /etc/logrotate.d/picasauploader
and add:
/opt/var/log/picasaupload.log {
    rotate 4
    weekly
    compress
    delaycompress
    missingok
    notifempty
    prerotate
      while [ "`ps aux | grep picasauploader.jar | grep -v grep | wc -l`" = "1" ]
        do
          sleep 10
        done
    endscript
}


This will rotate your picasauploader logs once a week and keep at least 4 weeks worth of logs. It is easy to modify these parameters in the config file above.

As logrotate is started by the same cron that starts the picasauploader (daily cron), you will notice that I try to make sure the picasauploader is done before rotating the logs...

Don't forget to change the path if your log is somewhere else!

10 comments:

  1. this looks good, i am gonna try it on my synology NAS.
    any chance of adding video upload as well? i know googlecl allows that, and quickly checking your code i think it shouldn't be much of an issue

    ReplyDelete
    Replies
    1. I haven't tested but it seems like it is just a matter of setting the right mime type in the upload:
      http://code.google.com/apis/picasaweb/docs/2.0/developers_guide_protocol.html#PostPhotoWithMetadata

      I won't be able to look at it soon... if you do it yourself, please send me the updated code!
      good luck!

      Delete
  2. This is fantastic! I was also thinking of using the googleCL tools but decided to do a little bit of searching to see if someone had done something similar in the past. I hate having to have picasa open on my windows machine to sync folder and images while I am processing the photos in Lightroom. I even tried running Picasa as a windows service using sc.exe but that was a failure lol. With this I wont even need anything running on my windows box as I can set this up on my NAS which runs freebsd. Thanks so much!

    ReplyDelete
  3. I'm having a problem when I run the java command. I keep receiving the following error:

    Exception in thread "main" java.lang.NumberFormatException: null
    at java.lang.Integer.parseInt(Integer.java:443)
    at java.lang.Integer.valueOf(Integer.java:570)
    at com.vincesoft.picasauploader.PicasaUploader.main(PicasaUploader.java:71)

    Any thoughts? It's driving me crazy.

    ReplyDelete
  4. Thanks for the JAR. Although useful I found it ate memory on my NAS, and so I had a bash at recording in python which seems slightly kinder to a low memory machine.

    It's here if anyone finds it useful (it features fully configurable two way sync)

    https://github.com/leocrawford/picasawebsync

    ReplyDelete
  5. Very nice! I am running this in Windows, and it works great!

    ReplyDelete
  6. Hey, do you think you could idiot proof this for me a bit more? You said it looks to see if there's a new folder and if there is it uploads. How can I set this baby up to upload if theres any new pics in the same folder? So basically it looksin a folder and if theres any new photos it uploads them. I have a user album on my site and all the images go into one directory.

    ReplyDelete
  7. Fantastic code - running it with cron on Ubuntu server.

    Any chance of adding video sync/upload functionality? or does it support certain formats already?

    ReplyDelete
  8. This tool worked fine on my Synology NAS until I upgraded the NAS to DSM 5.2, now I get following error :
    Exception in thread "main" com.google.gdata.util.AuthenticationException: Error authenticating (check service name)
    at com.google.gdata.client.GoogleAuthTokenFactory.getAuthException(GoogleAuthTokenFactory.java:628)
    at com.google.gdata.client.GoogleAuthTokenFactory.getAuthToken(GoogleAuthTokenFactory.java:500)
    at com.google.gdata.client.GoogleAuthTokenFactory.setUserCredentials(GoogleAuthTokenFactory.java:346)
    at com.google.gdata.client.GoogleService.setUserCredentials(GoogleService.java:362)
    at com.google.gdata.client.GoogleService.setUserCredentials(GoogleService.java:317)
    at com.google.gdata.client.GoogleService.setUserCredentials(GoogleService.java:301)
    at com.vincesoft.picasauploader.PicasaUploader.main(PicasaUploader.java:77)

    I'm using JAVA version 1.8.0_33

    Anybody an idea how to solve this ?

    ReplyDelete
  9. I think I found the reason why the authentication fails:
    Google stopped supporting oauth 1.0 api, and programs are now required to use oauth 2.0 api. (https://developers.google.com/identity/protocols/OAuth_ref#migration) It was the coincidence that I just upgraded my NAS on the same day as Google stopped using oauth 1.0.
    So my question to the developer of this great tool : is it possible to upgrade the tool with oauth 2.0 ?

    ReplyDelete