Thursday, January 26, 2012

How to install rtorrent with ip filtering into your Iomega StorCenter ix4-200d


EDIT: this post is outdated now, please see this post instead with a much newer version of rtorrent that works with magnets and natively supports ip filtering

This tutorial uses unsupported features of the IOMEGA Storcenter ix4-200d. It worked for me but use it at your own risk! It should work (again, it is unsupported) on the ix2 Storcenter as well.
Tutorial tested on IOMEGA Storcenter ix4-200d firmware 3.1.14.995
The torrent software supplied with the Storcenter doesn't work well for me: some torrents never load, some disappear etc.. plus there is no ip filtering capability. The aim of the tutorial is to install rtorrent on the NAS which seems the most logical choice for a NAS (light weight and reliable) and explains how to enable ip filtering directly within rtorrent which is especially usefull since peerguardian/moblock can't be installed on the NAS because some kernel modules are missing...


1. SSH into your NAS
See my other post: How to ssh into your Iomega StorCenter ix4-200d


2. Install the software
See my other post here to setup at the minimum ikg and ipkg-opt. Then:
ipkg-opt install rtorrent
ipkg-opt install lighttpd
ipkg-opt install screen

If you don't want to connect remotely to rtorrent to manage it from you computer, you can skip the rest of this section...
Install nTorrent on your computer http://code.google.com/p/ntorrent/
Install xml-rpc on the NAS:
ipkg install optware-devel
ipkg install libcurl-dev
cd /opt/tmp/
svn checkout http://xmlrpc-c.svn.sourceforge.net/svnroot/xmlrpc-c/stable xmlrpc-c    
cd xmlrpc-c/
./configure --prefix=/opt
make
make install
Note: ou can choose something other that nTorrent. Please give me you feedback in the comments if you do.


3. Configure the software
Fix paths in the startup script:
vi /opt/etc/init.d/S99rtorrent
Some absolute paths need to be set. Replace screen with /opt/bin/screen in lines like:
su -c "/opt/bin/screen -ls | grep -sq "\.${srnname}[[:space:]]" " ${user} || su -c "/opt/bin/screen -dm -S ${srnname} 2>&1 1>/dev/null" ${user} | tee -a "$logfile" >&2
except:
su -c "/opt/bin/screen -S ${srnname} -X screen /opt/bin/rtorrent ${options} 2>&1 1>/dev/null" ${user} | tee -a "$logfile" >&2
replace:
if ps | grep -sq ${pid}.*rtorrent ; then # make sure the pid doesn't belong to another process
with
if ps auxxx | grep -sq ${pid}.*rtorrent ; then # make sure the pid doesn't belong to another process
Configure rtorrent (you should get a real rtorrent help for this, I am just trying to get you to a point where is works!):
vi /opt/etc/rtorrent.conf
set download and torrent directories:
instead of
directory = /opt/share/torrent/work/
set something like
directory = /mnt/pools/A/A0/torrents/rtorrent/download
instead of
schedule = watch_directory,5,5,load_start=/opt/share/torrent/dl/*.torrent
set something like
schedule = watch_directory,5,5,load_start=/mnt/pools/A/A0/torrents/rtorrent/torrents/*.torrent
comment out:
#schedule = untied_directory,5,5,stop_untied=
add at the end:
#files rwx for everybody
system.set_umask = 0000
Run:
mkdir /opt/share/torrent/session
Then, create the directories rtorrent/download and rtorrent/torrents inside the torrent share using regular NAS access (to have the right permissions)


4. Configure the software for remote access
This is only if you want to manage your rtorrent remotely:
Thanks to http://www.nslu2-linux.org/wiki/HowTo/RtorrentWithRemoteGUI for the setup.
Security warning: if you follow these steps, anybody that can access port 8081 of you NAS will be able to send commands to rtorrent! You want to make sure that this port is only accessible from your local network.
vi /opt/etc/lighttpdlighttpd.conf
between
#                               "mod_rrdtool",
and
"mod_accesslog" )
add
"mod_scgi",
and at the end add:
scgi.server = (
"/RPC2" => ( 
    "127.0.0.1" => (
        "host" => "127.0.0.1",
        "port" => 5000,
        "check-local" => "disable"
        )
    )
)
vi /opt/etc/rtorrent.conf
and at the end add:
scgi_port = localhost:5000


4. Test your setup
/opt/bin/rtorrent -n -o import=/opt/etc/rtorrent.conf
if you get:
rtorrent: Fault occured while inserting xmlrpc call.
did you install xmlprc correctly? is ld.so.conf updated correctly? did you run ldconfig?

to connect to the running instance:
/opt/bin/screen -r rtorrent
and kill the terminal (putty) to exit or press Ctrl-a d.

For remote access: you can start lighthttpd on the NAS
/opt/etc/init.d/S80lighttpd start
and then start nTorrent on your computer and connect to your NAS port 8081 (by default) on path /RPC2.



5. Get rtorrent to start automatically on reboot
Follow the tutorial How to run a program at boot on Iomage Strocenter You just need to add the following lines to the script:
/opt/etc/init.d/S80lighttpd start >> /opt/init-opt.log
/opt/etc/init.d/S99rtorrent start >> /opt/init-opt.log
If you have another brand of NAS (or a regular linux OS), just try to link the startup scripts to /etc/rc2.d/ like ou would normally do an a linux box:
ln -s /opt/etc/init.d/S80lighttpd /etc/rc2.d/S80lighttpd
ln -s /opt/etc/init.d/S99rtorrent /etc/rc2.d/S99rtorrent


6. Get a peerguardian like protection
First, I tried to install peerguardian linux but ran into a wall: the LifeLine OS on the Iomega Storcenter does not have the right kernel modules. I tried to recompute the kernel from the sources given by IOMEGA on their website (it is available for download in the support section) I got it to compile but insmod of the required module (x_tables.ko) does freeze the kernel (hard reboot required). Since I could not think of a safe way to push further in this direction (without risking to brick the NAS), I investigated other possibilities...(post a comment if you want more details on the kernel compilation)
I thought about abandoning rtorrent altogether and try Vuze (which has ip filtering).I got it to run but it was pretty unstable (jre crash)...

Luckily, someboby wrote a patch for rtorrent so that it supports ip filtering. I got it to compile on my NAS (version 0.8.6) and here is the result.You just need to:
cd /opt/bin/
mv rtorrent rtorrent.sav
wget http://dl.dropbox.com/u/50398581/rtorrent-0.8.6/rtorrent
that will give you a rtorrent with ip filtering supported.
Note: this only work if you previously installed version 0.8.6 of rtorrent!

The precompiled version will only work if you have an "armel" architecture. Otherwise, you need to recompile from source (see point 7 below)

The some config:
vi /opt/etc/rtorrent.conf
and add at the end
ip_filter=/mnt/pools/A/A0/torrents/rtorrent/ipfilter/level1
schedule = filter,18:30:00,24:00:00,reload_ip_filter=
thanks to http://bogdan.org.ua/2011/04/01/rtorrent-enhanced-with-ipfilter-and-geoip-debian-squeeze-amd64-package.html

Now, we need to download and update regularly the ip filter file:
vi /etc/cron.daily/rtorrent_ipfilter
#!/bin/sh
cd /mnt/pools/A/A0/torrents/rtorrent/ipfilter/
wget http://list.iblocklist.com/?list=bt_level1
mv index.html\?list\=bt_level1 level1new.gz
gunzip level1new.gz
rm level1
mv level1new level1
then:
mkdir /mnt/pools/A/A0/torrents/rtorrent/ipfilter/
cd /etc/cron.daily/
chmod a+x rtorrent_ipfilter
./rtorrent_ipfilter
That's it: you just need to restart rtorrent to enjoy ip filtering. The ip filter file will be update everyday thanks to the cron (and rtotorrent will reload it).




7. In case you want/need to compile rtorrent with ip filtering yourself!
This is usefull if you are compiling a different version or a different architecture (please comment and report your success if you do so).

First, you need the header files for libsigc++-2.0.
wget http://ftp.de.debian.org/debian/pool/main/libs/libsigc++-2.0/libsigc++-2.0-dev_2.0.18-2_armel.deb
dpkg --instdir=/opt/ --admindir=/opt/dpkg/ -i libsigc++-2.0-dev_2.0.18-2_armel.deb

Then, take care of litorrent. I recompile libtorrent to install the correct headers as I don't find them anywhere (I didn't look for a deb archive with the correct headers but that might have done the trick...):
wget http://libtorrent.rakshasa.no/downloads/libtorrent-0.12.6.tar.gz
tar -xvf libtorrent-0.12.6.tar.gz
cd libtorrent-0.12.6
PATH=$PATH:/opt/bin
Then:
vi configure
and add at the begining of the configure script:
OPENSSL_CFLAGS='-I/opt/include/'
OPENSSL_LIBS='-L/opt/lib/ -lopenssl'
STUFF_LIBS='-L/opt/lib/ -lsigc-2.0'
STUFF_CFLAGS='-I/opt/usr/include/sigc++-2.0/ -I/opt/usr/lib/sigc++-2.0/include'
Note that I do edit the configure because I don't manage to get ipkg-config to work correctly. Using /opt/bin/pkg-config works better than the /bin/pkg-config but still not good enough...
./configure --prefix=/opt/
make
make install

Then, the main thing: rtorrent
ipkg-opt install libcurl-dev
ipkg-opt install ncurses-dev
PATH=$PATH:/opt/bin
wget http://libtorrent.rakshasa.no/downloads/rtorrent-0.8.6.tar.gz
tar -xvf rtorrent-0.8.6.tar.gz
cd rtorrent-0.8.6
vi configure
and add at the begining of the configure script:
sigc_LIBS='-L/opt/lib/ -lsigc-2.0 -L/lib/'
sigc_CFLAGS='-I/opt/usr/include/sigc++-2.0/ -I/opt/usr/lib/sigc++-2.0/include -I/opt/include/ncurses'
libcurl_LIBS='-L/opt/lib/ -lcurl'
libcurl_CFLAGS='-I/opt/include/'
libtorrent_LIBS='-L/opt/lib/ -ltorrent'
libtorrent_CFLAGS='-I/opt/include/'
I did not know were to put the ncurses include, that's why you'll find it in sigc_CFLAGS...

Now, install the patch to have ip filtering (more details on the patch here http://libtorrent.rakshasa.no/ticket/239):
wget http://libtorrent.rakshasa.no/raw-attachment/ticket/239/rtorrent-0.8.6-ip_filter_no_boost-fast-bsd2.patch
/opt/bin/patch-patch -p1 < rtorrent-0.8.6-ip_filter_no_boost-fast-bsd2.patch
./configure --prefix=/opt/ --with-xmlrpc-c=/opt/bin/xmlrpc-c-config
Then, I got the following issue:
/opt/arm-none-linux-gnueabi/lib/libdl.so.2: undefined reference to `_dl_tls_get_addr_soft@GLIBC_PRIVATE'
My system is starting to be a mess. The problem comes from the fact that I have 2 libdb.so libs:
root@xxx:/opt/tmp/rtorrent-0.8.6# ls -l /mnt/apps/lib/libdl.so.2
lrwxrwxrwx 1 root root 12 Sep  9 20:46 /mnt/apps/lib/libdl.so.2 -> libdl-2.8.so
root@xxx:/opt/tmp/rtorrent-0.8.6# ls -l /mnt/system/opt/arm-none-linux-gnueabi/lib/libdl.so.2
lrwxrwxrwx 1 root root 12 Jan  6 22:47 /mnt/system/opt/arm-none-linux-gnueabi/lib/libdl.so.2 -> libdl-2.5.so
to fix the issue:
rm /mnt/system/opt/arm-none-linux-gnueabi/lib/libdl.so.2
Then, compile and install:
make
make install
final issue:
Could not compile XMLRPC-C test.
This one came from a mismatch between heders and libs. Recompiling the package xml-rpc from source did fix the issue. 8. What's next In a different post, I will detail how to install Vuze headless (without graphical interface)... I don't recommand installing vuze because I ran into stability issues while testing it (several jre crashes). However, a new Jre version might solve the issue...
On top of that, you can't do much from the Web UI: when you want to setup something, you ofen have to use the Vuze command line inteface and I did not find any proper documentation for it.
Note as well that rss feeds features don't work in headless mode.

Monday, January 23, 2012

How to run a program at boot on the Iomega Storcenter NAS


This tutorial uses unsupported features of the IOMEGA Storcenter ix4-200d. It worked for me but use it at your own risk! I undertand it works (but still isn't unsupported by IOMEGA) on the ix2 Storcenter as well.
Tutorial tested on IOMEGA Storcenter ix4-200d firmware 3.1.14.995
This post is extracted from a previous post (how to install crashplan on Iomega storcenter). I am planning more tutorials on the Iomega storcenter ix4 and wanted centralize this part in case it needs to evolve with future firmwares...


1. Enable SSH on the NAS
see my other post How to SSH into your Iomega Storcenter




2. Create a script that runs at boot
Iomega OS (EMC LifeLine) does not respect what is inside /etc/rcx.d/. If you have another brand of NAS, chances are that installing the script into /etc/rc2.d/ will result in the script running at boot...
If you already downloaded the scripts to have a command run at boot, you can just add the new command below the one you already have in /opt/init-opt.sh. Just make sure the commands all return immediately (or add a & at the end) so that the script does not get stuck before reaching the last command.

If that the first time you do this, download the scripts:
cd /opt/
wget http://dl.dropbox.com/u/50398581/Storcenter%20add%20on%20boot/editconfig.sh
chmod +x /opt/editconfig.sh
wget http://dl.dropbox.com/u/50398581/Storcenter%20add%20on%20boot/init-opt.sh
chmod +x /opt/init-opt.sh
Now we start editing the XML list of programs that will automatically be started. Run:
/opt/editconfig.sh
You will see lots of Groups. We are going to add one <program> to <group level="1">. We will add:
<program name="init-opt" path="/opt/init-opt.sh">
<sysoption restart="-1"/>
</program>
the scripts content for you reference:
/opt/editconfig.sh
#!/bin/sh
# edit the bootup config of the ix-2
# inspired by http://www.chrispont.co.uk/2010/10/allow-startup-daemons-on-storcenter-ix2-200-nas/
mknod -m0660 /dev/loop3 b 7 3
chown root.disk /dev/loop3
mkdir /tmp/apps
mount -o loop /boot/images/apps /tmp/apps
vi /tmp/apps/usr/local/cfg/sohoProcs.xml
sleep 1
umount /tmp/apps
rm /dev/loop3
/opt/init-opt.sh
#!/bin/sh
# modified from http://techmonks.net/installing-transmission-and-dnsmasq-on-a-nas/
rm /opt/init-opt.log
echo "Last bootup:" >> /opt/init-opt.log
date >> /opt/init-opt.log
#Add your command below
#/etc/init.d/xxxxxx start >> /opt/init-opt.log
while true; do
 sleep 1d
done
Then, you just need to edit editconfig.sh add the command(s) you wish to run after the
#/etc/init.d/xxxxxx start >> /opt/init-opt.log
line!

How to install software into your Iomega StorCenter ix4-200d


This tutorial uses unsupported features of the IOMEGA Storcenter ix4-200d. It worked for me but use it at your own risk! It should work (again, it is unsupported) on the ix2 Storcenter as well.
Tutorial tested on IOMEGA Storcenter ix4-200d firmware 3.1.14.995
The aim of the tutorial is to be able to add programs to you NAS without having to go too deep in the system. This is also helpful to compile natively on the NAS without needing to cross compile for your architecture....


1. SSH into your NAS
See my other post: How to ssh into your Iomega StorCenter ix4-200d


2. Directory Structure on the NAS
The Lifeline OS (Iomega's OS) does put most of the root file system in read only mode. It is not much use to try to put stuff there anyway because the partitition is very small
You can type:
df
Filesystem           1K-blocks      Used Available Use% Mounted on
rootfs                   51200      5652     45548  12% /
/dev/root.old             6613      2119      4494  33% /initrd
none                     51200      5652     45548  12% /
/dev/md0_vg/BFDlv      4128448    619496   3299240  16% /boot
/dev/loop0              587077    580124      6953  99% /mnt/apps
/dev/loop1                4959      2230      2473  48% /etc
/dev/loop2                 216       216         0 100% /oem
tmpfs                   255748         0    255748   0% /mnt/apps/lib/init/rw
tmpfs                   255748         0    255748   0% /dev/shm
/dev/mapper/md0_vg-vol1
                      16775168   3283704  13491464  20% /mnt/system
/dev/mapper/2602b0ce_vg-lv43ec31bd
                     2867212288 1169119852 1698092436  41% /mnt/pools/A/A0
to see the partitions and their mountpoint.
The idea seems to be that third party programs should be installed in the /opt/ directory, which has ample storage (16GB) whereas root (/) only has 50MB.


3. Use ipkg
ipkg is intalled by default in the Iomega storcenter. We just need to specify the right place to find the packages:
vi /etc/ipkg.conf
src cross http://ipkg.nslu2-linux.org/feeds/optware/cs08q1armel/cross/stable
src cross http://ipkg.nslu2-linux.org/feeds/optware/cs08q1armel/cross/unstable
Then type:
ipkg update
to build the list of available packages.


The problem of this setup is that you won't be able to install some packages because installation will fail because part of the filesystem is readonly.
Thanks to ipkg, there is an easy fix:
ipkg install ipkg-opt
This installs the binary /opt/bin/ipkg-opt. The idea is then to use this binary instead of the regular ipkg: as a result all packages will be installed in /opt/ and you won't run into problems with the read only filesystem.
The only drawback is that /opt/bin/ is not in your path... There is a simple remedy for that:
PATH=/opt/bin:$PATH
Note: this is not persistent (if you start another shell, you will need to do that again).
Also, as a one time persistent thing, I recommand to do
vi /etc/ld.so.conf
and add
/opt/lib/
at the end. That's the main problem with /opt installed software: you might end up to get duplicated libraries between /lib and /opt/lib (ldd and ldconfig are your friends).

You also need to do:
mv /opt/etc/ipkg.conf /opt/etc/ipkg.conf.old
ln -s /etc/ipkg.conf /opt/etc/ipkg.conf
so that you config in /etc/ipkg.conf remains useable with /opt/bin/ipkg and /opt/bin/ipkg-opt

Then type:
/opt/bin/ipkg update
to setup the list of available packages for /opt/bin/ipkg /opt/bin/ipkg-opt

4. Install utilities and optware-devel
First install the utilities you miss to do some actual linux stuff:
ipkg-opt install zip unzip bzip2 gzip

If you want a full gcc toolchain to compile your own applications from source.
ipkg-opt install optware-devel
The compilation can be slow but this allows you to natively compile on your NAS (I think it is simpler because there is no need to set up cross compiling on another box)...

5. Install armel/debian compiled software
Unfortunately, you will soon discover that some of the packages you want are not available for ipkg.
You can then either compile your own software (see next point) or get some ready-made debian archives....
In this case, I suggest to use the following command (for example for libsigc++-2.0-dev):
cd /opt/tmp/
wget http://ftp.de.debian.org/debian/pool/main/libs/libsigc++-2.0/libsigc++-2.0-dev_2.0.18-2_armel.deb
dpkg --instdir=/opt/ --admindir=/opt/dpkg/ -i libsigc++-2.0-dev_2.0.18-2_armel.deb
Note: do not use /tmp/ as the space available there is very small...
Note2: be careful to choose packages compiled for your architecture (armel in my case)! The above command will install your soft as if /opt/ was the root directory (you will end up with /opt/usr/lib directories and the like). As a result, you might need to add stuff in your PATH or edit /etc/ld.so.conf.
Be careful not to make a mess of your system or you will soon end up with several times the same library (with different versions) at different locations... You will need to sort this manually(ln, rm...)


6. Compile from source
For example, a very classic install for libnfnetlink:
cd /opt/tmp/
wget http://www.netfilter.org/projects/libnfnetlink/files/libnfnetlink-1.0.0.tar.bz2
tar -xvf libnfnetlink-1.0.0.tar.bz2
cd libnfnetlink-1.0.0
PATH=$PATH:/opt/bin
./configure --prefix=/opt/
make
make install
Note: to get bzip2 to work I had to do before the tar -xvf:
ln -s /opt/bin/bzip2-bzip2 /opt/bin/bzip2
Another example using svn
cd /opt/tmp/
PAH=/opt/bin:$PATH
svn checkout http://xmlrpc-c.svn.sourceforge.net/svnroot/xmlrpc-c/stable xmlrpc-c    
cd xmlrpc-c/
./configure --prefix=/opt
make
make install
Don't forget the --prefix=/opt to specifiy you want to install your package.

When compiling from source, you run into the usual complation problems you can get with linux (libraries/includes not found etc...). It gets even more annoying because default stuff does not work well anymore (package manager is not where expected etc), and sometimes you end up having to specify the complie flags yourself.
For example, I recently had to edit the configure script of a source tarball to add:
sigc_LIBS='-L/opt/lib/ -lsigc-2.0 -L/lib/'
sigc_CFLAGS='-I/opt/usr/include/sigc++-2.0/ -I/opt/usr/lib/sigc++-2.0/include -I/opt/include/ncurses'
libcurl_LIBS='-L/opt/lib/ -lcurl'
libcurl_CFLAGS='-I/opt/include/'
libtorrent_LIBS='-L/opt/lib/ -ltorrent'
libtorrent_CFLAGS='-I/opt/include/'
-dev packages can be difficult to find with ipkg, this is where you often need to get a .deb package or compile the library from source just to get the header files right...

7. Conclusion
As you noticed, it is just a matter of using the tools (and using them right). It just gets a little bit more complicated because the usual package manager does not work out of the box, the procedure is unsupported by the hardware vendor and precompiled packages can be difficult to find for armel...

Thursday, January 12, 2012

How to ssh into your Iomega StorCenter ix4-200d


This tutorial uses an unsupported feature of the IOMEGA Storcenter ix4-200d. It worked for me but use it at your own risk! I undertand it works (but still isn't unsupported by IOMEGA) on the ix2 Storcenter as well.
Tutorial tested on IOMEGA Storcenter ix4-200d firmware 3.1.14.995
This post is extracted from a previous post (how to install crashplan on Iomega storcenter). I am planning more tutorials on the Iomega storcenter ix4 and wanted centralize this part in case it needs to evolve with future firmwares...


1. Enable SSH on the NAS
Go to
http://your-nas-adress/diagnostics.html
click "enable SSH"
on older firware versions, I understand an equivalent page could be found at:
http://your-nas-adress/support.html

If the admin password of the NAS is "pass" the root password to use in ssh is sohopass.(thanks http://planetkris.com/2010/05/iomega-storcenter-ix2-ssh-email-notifications-and-busybox-init-d)