CH3SNAS: fun_plug and nzbget

Date October 12, 2008

Today was testing day with Andrei (developer of nzbget) and Fonz (developer of FFP).  Cause nzbget crashed during repair of RAR files after checking the validation of the RAR files.

Finally it resulted in a new binary created by Andrei. (no fix from Fonz yet cause it’s still unclear what is the real issue or if it has to do with statically and dynamically linking of libraries).

So for all users of the Conceptronic CH3SNAS or DNS-323 who do not want to use optware (ipkg) (like me) but want to use NZBGET with FFP 0.5

Here are the guidelines:

First make sure you are using the correct extra libraries libpar2 (libpar2-0.2-2.tgz) (install with funpkg -i after download) and libsigc libsigc++-2.0.18-1.tgz (install with funpkg -i after download)

(according to Andrei it might even not be necessary to need these seperate tools when using the special compiled version for CH3SNAS/DNS-323).

You can find the source & scripts here

Make sure you download and install the correct unrar version. I have just upgraded my version to 3.7.8-1 from ULI the older version 3.4 caused strange crash during a long unrar session.

Download the nzbget-static-all-release.zip file, and if you have troubles you could use the debug version. Also available for download from the same webpage. This is the modified binary.

Put this file on your CH3SNAS. unzip this file and make sure it will be in /ffp/bin
You can use my nzbget.conf script, modify the usenet server details!

the nzbget.conf file must be put in /ffp/etc  folder
All my files are in /mnt/HD_a2/usenet
and under this folder there are 4 other folders needed: nzb, download, queue and tmp

If you want to process the RAR files after download you need a postprocess.sh file, you can grab your copy from my page and make sure it is called postproces.sh and put in /ffp/etc as well. Modify pathnames if necessary.

The last thing you need to do is grab the nzbget.sh file and put it in /ffp/start and make sure you make it executable with: chmod a+x nzbget.sh

You can check the working witht the command: sh nzbget.sh start - check it with sh nzbget.sh status - and stop it with sh nzbget.sh stop

Reboot your CH3SNAS or DNS-323 and after a few minutes you should be able to enter telnet again and see that nzbget is running (use ps as command to see the processes).

Smart people also make lighttpd working in combination with nzbgetweb .. to get a webfrontend for nzbget. the only thing you have to do is copy a nzb file into the /mnt/HD_a2/usenet/nzb folder

update 13-10:

changed links to folder content so that updates can be found rather than missing.

update 16-10:

Oeps.. some links to the files were gone .. Note: I am busy to get a more advanced postprocess script working ok.

When using NZBGETWEB

Addon: uploading large NZB > 8MB through the webinterface of NZBGETWEB is not possible if you do not use a separate PHP.INI. So please copy from /ffp/etc/examples the php.ini-recommended to /ffp/etc/php.ini, change the max upload file size. Although lighttpd has PHP support it does not use PHP.INI so you have to add this. open /ffp/etc/lighttpd.conf and edit : “bin-path” => “/ffp/bin/php-cgi“, to  /ffp/bin/php-cgi -c /ffp/etc/php.ini“; restart lighttpd and also large NZB files can be uploaded with NZBGET if you use this webfrontend.

enjoy!

50 Responses to “CH3SNAS: fun_plug and nzbget”

  1. Nasp said:

    Well, i use nzbget from past 3 or 4 month, but the ch3snas can repair and extract the rar archives ?

  2. Dennis said:

    yes, nzbget is doing par itself en unrarring you need a postscript.sh and unrar ..
    My test the last 2 days was 1 crash during unpacking 28GB, but it might an issue of my unrar (still testing) .
    and further it parred and unrarred about 80GB without issue ..

  3. Ruisbak said:

    Nice tut,

    One typo to correct:

    - the nzbget.conf folder must be put in /ffp/etc
    change to:
    - the nzbget.conf script must be put in /ffp/etc

    I must also type: /mnt/HD_a2/ffp/bin# chmod a+x nzbget
    because nzbget was not executable in my case.

    The postprocess.sh file needs to have the full path to unrar, otherwise it doesn’t seem to work.
    UnrarCmd=/mnt/HD_a2/usr/local/bin/unrar

    Also the ~ needed to be removed in the $MAINDIR=/mnt/HD_a2/usenet
    statement in nzbget.conf

    Hope this helps to make it even more perfect. :-)

    P.S. Why are you using a very old version of unrar?
    For optware unrar_3.7.8-1_arm.ipk is already available.

  4. Dennis said:

    Ruisbak
    made some changes
    I do not use optware, but I upgraded my rar this week. I put all my binaries in /ffp/bin cause that is also in my PATH statement so I do not have to use a complete path to unrar ;-)

    I also modified the $MAINDIR part, but not tested if it is ok in my environment (but I think so) ..

  5. Ruisbak said:

    Sure, putting unrar in /ffp/bin also fixes the problem but because the tut doesn’t say anything about this you have to tell.

  6. Dennis said:

    check the special unrar version compiled by Uli .. ;-)

  7. Bert said:

    Hi Dennis,

    Where can I get the “postprocess.sh” file?

  8. Chris said:

    Thanks for the installation instructions. Installation was simple and got it running in no time.

    However, I’ve created a NZB file using binsearch (5,7 MB) which downloaded lots of classic Dr Who series. Unfortunately, some RAR’s were corrupt, so the postprocess script tries to repair stuff using the par libs.

    I’ve got at least one RAR file that it does not want to repair. The NZBGET daemon crashes, dumps its core and halts. I’ve installed all of the libs and scripts you’ve posted, but nevertheless it breaks during repair. I’ve got a couple of core.#### files in my download directory. Anyone a clue how to get things in order?

    p.s. Dennis: ik vind dat je goed bezig bent!!! Door jouw betrokkenheid bij de Conceptronic productlijn heb ik een CH3SNAS gekocht en denk nu dat ik een CFULLHDMA ga halen ;-) Keep up the good work!!!

  9. dennis said:

    Hi Chris ..
    dank je ;-)

    de vraag die ik heb is waar je de nzbget vandaan hebt. nzbget crashed namelijk als je de versie download van de maker of van Fonz (op PAR niveau)
    pak de versie die op mijn site staat, die heeft meer dan 150GB in 2 dagen gedownload en toen begon de NAS bijna te roken (figuurlijk) ;-)

    http://www.aroundmyroom.com/ch3snas-funplug

    Dus even nieuwe versie installeren (denk ik)

  10. Ruisbak said:

    Thx, looking forward to your “more advanced postprocess script working ok”.

    One thing on the nzbget.conf file, please make the default server connections 2 and not 4.

    # Maximal number of simultaneous connections to this server
    Server1.Connections=2

    Lots of people are not allowed to make more than 2.
    This generates unwanted denied access messages.
    We want to use this bandwith for leeching. ;-)

  11. dennis said:

    4 is standard in the nzbget.conf .. I did not change anything ..
    Next to that: for usenet people use OFTEN a payed usenet server due to the retention and completeness of the binaries.
    So I will not change that ;-) ..

  12. Chris said:

    Hoi Dennis,

    ik ben erachter gekomen dat ‘ie stiekem de nzbget uit optware gebruikte. Ik heb de “oude” weggegooid en jouw zipfile in /ffp/bin gezet. Daarna deed ‘ie het als een speer!

    (grmbl… wat stom dat ‘ie de optware versie gebruikte, ik heb kennelijk met m’n neus zitten kijken tijdens de initiele installatie ;-)

    Thanks for the support!

    Chris

  13. Nasp said:

    And NZBGET with SSL suport, anyone have test ?

  14. dennis said:

    Andrei has compiled the CH3SNAS version with SSL support afaik ..

  15. Chris said:

    Hoi Dennis,

    de “nieuwe” nzbget heeft weer een tijdje gedraaid, maar stopt er nu toch weer mee tijdens het verifieren van een RAR bestand. De “andere” nzbget stopte ermee tijdens het repareren.

    Heb je nog suggesties? Alvast bedankt!

  16. dennis said:

    zorg ervoor dat je geen enkele andere oude versies op de bak hebt staan

    ik heb 150GB gewoon zitten binnen te halen met vele PAR sessies..
    (En nog steeds)

    Oh let erop: gebruik wel een GOEDE versie van UNRAR !!

  17. Nasp said:

    Works good here too, full tested! Par/repaired is a litle slow, but unrar is more fast.

    Sameone have test it Par/repair and download queue not paused ?

    Also i cant make nzbget works it SSL with my news ( xlnet ), i have made all the possibilities in nzbget.conf.

  18. Ruisbak said:

    All seems to be working great.

    Only “problem” is that HDD’s don’t spin down anymore.
    Any ideas for this?

  19. Dennis said:

    issue can be related to the fact that nzbget is watching for nzb files every x minutes / seconds

  20. Nasp said:

    for this i think it is better to put ffp and nzbget, etc… runing from an usbpen, it the nzb folder in the pendrive.

  21. Ruisbak said:

    OK, I’ll change the watch time for nzb files.
    See how that works.
    With ffp only installed, HDD spindown works fine.
    So it must be nzbget.

  22. Ruisbak said:

    Yep, that did the trick.
    In nzbget.conf the line “NzbDirInterval=5″ changed to “NzbDirInterval=3600″

    There is also this comment: “# Value “0″ disables the check.”
    Might be interesting but how do you start leeching if it never checks for nzb’s?

  23. dennis said:

    he he he ;-)
    no idea ;-)

  24. Sayonara said:

    Thx for the only tutorial on the internet how to get nzbget working on CH3SNAS :)
    I configured the script nicely except for the unrar thing. do we need to use the UNPAK.TAR file ?

    Now I would like to use it with my webbrowser and lighttpd. Is there some usefull information you can provide me with ?

    Groeten :)

  25. dennis said:

    unpak.tar contains the postprocess script for nzbget to have the rar files unpacked
    it has my modified script for use with ch3snsas and the original script if you do not like my changes (more or less only pathnames)

    if you want to use your webbrowser and lighttpd, than please proceed installing lighttpd with php support and download nzbgetweb . .that is almost self explanatory
    but if needed I can put my config settings of nzbwebget.

  26. Nasp said:

    PROBLEMS . i have same problems it postprocess, delete the rars and the iso of same of dual-layers isos that i dowload…. damm!!!

    because of that postprocess dont delete my rar files, i delete manual!!

  27. dennis said:

    NASP .. I do not understand?

    the postprocess script I use is ‘perfect’ as long as you do not use 1 NZB with multiple movies

  28. Nasp said:

    Yes… Dennis

    For me the Postscript, have deleted 2 of .iso ( dualLayers isos ) with the rars files, extract and than deleted all files rars and isos. The folder name and the files .nfo etc are intact.
    Maybe the extract it is not sucess, and deleted the rar files.

    With other downloads files ( i have download more than 50gb in the past days ) works good.

    I have to download again…. ( For this problem i have altered the postprocess.sh for not deleted rar files )

  29. Eric said:

    Installed all as in tutorial, but uploading does not work for me in NZBGETWEB. worked exactly three times and now i get error 413 or just an attempt to open the status.php page. I am using the modified php.ini (size is 8mb) Somebody any thoughts?

  30. Dennis said:

    you might give me a lead to the nzb file ;-) than I can test my config

    Yesterday After downloading a large collection of RAR’s the device crashed (after downloading and unrariing more than 100GB ..

    I restart it and check what will happen with next file. Might be memory or something else ..

  31. Chris said:

    I’ve downloaded more than 80GB at this moment, not without trouble (see other postings). But now it downloaded approximately 60GB without any issues.

    I managed to get it stable/”stablish” by using the debug.testing.0.60 version and lowering the max. threads to only 10 (instead of 50). I do not know what this setting really does, and whether the setting is actually being used by nzbget. But hey - it worked for me. It pulls approx. 1500kb/s from my newsserver (which is through my connection pretty much the max).

    My CH3SNAS is a pretty busy device, as it runs lighttpd, PHP, nzbget, twonkymedia, nfs for a start. It is not unthinkable that my instability issues are caused by memory issues :-)

  32. Eric said:

    Dennis,
    I just tested an upload of an NZB from http://www.nzb-portal.nl “overig” Poker Sercret ebook (2kb). This one worked!? Then I tested from the same site “tv-series” SMALLVILLE (720P 5.1 DD NL.SUBS) S7D4 (698kb). This one failed. I removed the poker secrets from my queue and tried it again with the poker one. This still works, so it might be a size thing. But they are both well under 2M. Help…

    Downloading however is going great. Still using the “old” postprocess.sh. First want to get a feeling of what to expect before I change things again.

  33. dennis said:

    what are the settings in your php.ini and your config of lighttpd ?

  34. Eric said:

    Here is the changed part of my php.ini;
    ; Maximum allowed size for uploaded files.
    upload_max_filesize = 8M

    In my lighttpd.conf I have modified this;
    ## php support
    ## uncomment the following lines and the mod_fastcgi module above
    fastcgi.server = ( “.php” => ((
    “bin-path” => “/ffp/bin/php-cgi -c /ffp/etc/php.ini”,
    “socket” => “/tmp/php-cgi.socket”,
    “max-procs” => 2

    Thanks for all your efforts so far.

    EDIT: oeps, think i found it:
    in php.ini the upload folder is left to default. In my lighttpd.conf the default folder is tmp but it had never been created before. I now made the folder tmp and uploads seem to work.. Time for more testing

  35. Dennis said:

    yeah .. make sure the /tmp is somewhere on the disk like /mnt/HD_a2/tmp cause /tmp can be memory and if that is not present or available .. ;-)

  36. Nasp said:

    Damm

    I tried do update for nzbget 6.0 ( firts i have nzbget 5.1 - andrei version works good ins pasts months with unrar comand…etc ) and now nzbget dont works!! Also i returned to nzbget 5,1 and this version now dont work too!!!!

    I dont now what is, i installed the nzbget version by fonz, and than it works. ( Now only the fonz version works )

    /ffp/bin/nzbget -D ……….. and telnet stop

    telnet again /ffp/start/nzbget.sh status
    returned
    nzbget running as a pid 1659

    I have removed all ffp directory and fresh a new install, and nothing!! same problem.

    Also the nzbget.sh when i reboot my ch3snas for same reason i dont know but kill my lighttpd start, i know that because if i remove nzbget.sh the lighttpd runs well

    the ffp.log dont show me nothing!! and dont have a nzbget.log because he dont run.

    My last opcion format the drive?

  37. Nasp said:

    well

    tar xvzf nzbget-0.6.0-testing-r237-bin-dns323-arm.tar.gz in Volume 1, resolved the problem.

    damm, so much hours!!!

  38. dennis said:

    that’s the version I am using as well

  39. Seppie said:

    Dennis,

    Ive collected all of the required files for the nzbget and got it running but sometimes after and download i get:

    Thu Nov 13 07:57:10 2008 INFO Executing post-process-script for (filepath)
    and then it just stops. It doesn’t start the extraction part. (Doesn’t occure every time)

    Any ideas on this?

  40. Dennis said:

    @ Seppie yep .. I think so ..

    I think it has to do with a special SED setting in the unpak.sh

    check:

    #Add ‘\’ to regular expression metacharacters in a string.
    #Required so we can search for the string whilst using regualr expressions.
    # eg grep “^$string$”. this will fail if string contains ‘[].* etc.
    RE_ESCAPE_FOR_SED() {
    sed ’s/\([].[*/\]\)/\\\1/g’
    }

    # Same as RE_ESCAPE_FOR_SED but (|) are also meta characters.
    RE_ESCAPE_FOR_GREP() {
    sed ’s/\([].[*/\(|)]\)/\\\1/g’
    }

    http://apps.sourceforge.net/phpbb/nzbget/viewtopic.php?f=3&t=46&p=202&hilit=sed&sid=d6a9dd7be78e2c5d424f8a6ccf153c63#p202

    I changed the unpak.sh script with this and I have downloaded 160GB without issue.
    but I am still tracking it

  41. Skinberry said:

    Who can help me ??

    Just installed NZBget from this lovely site. Got it almost up and running, but there is a slight hickup:

    root@CH3SNAS:/mnt/HD_a2/ffp/start# sh nzbget.sh start
    Starting /ffp/bin/nzbget -D -c /ffp/etc/nzbget.conf
    /ffp/bin/nzbget: can’t load library ‘libstdc++.so.5′

    What am i missing ?

    Where can i find this libstdc++ file and do i need it ?
    This manual says nothing about it !

  42. dennis said:

    do you use fonz funplug 0.5 ?
    you need that.
    the libstdc++ is part of /ffp/lib

    /mnt/HD_a2/ffp/packages # locate libstdc
    /mnt/HD_a2/ffp/lib/libstdc++.a
    /mnt/HD_a2/ffp/lib/libstdc++.la
    /mnt/HD_a2/ffp/lib/libstdc++.so
    /mnt/HD_a2/ffp/lib/libstdc++.so.6
    /mnt/HD_a2/ffp/lib/libstdc++.so.6.0.8
    /mnt/HD_a2/ffp/lib/libstdc++_pic.a

    /mnt/HD_a2/ipkg/opt/lib/libstdc++.so
    /mnt/HD_a2/ipkg/opt/lib/libstdc++.so.5
    /mnt/HD_a2/ipkg/opt/lib/libstdc++.so.5.0.5
    /sys/crfs/usr/lib/libstdc++.so.5
    /sys/crfs/usr/lib/libstdc++.so.5.0.5

    Yes ipkg is installed but no use .. but it’s also in /sys/crfs

  43. Seppie said:

    Ive changed the SED setting but still getting:

    Fri, 14 Nov 2008 15:35:21 +0000 Executing post-process-script for Hancock.2008.1080p.5.1.ac3.Ps3.NL-Waspoeder/Hancock.2008.1080p.5.1.ac3.Ps3.NL-Waspoeder

    And then it just states as:
    Currently processing

    name stage % elapsed time
    Hancock 2008 1080p 5 1 ac3 Ps3 NL-Waspoeder executing script 2:33:01

    But nothing happens :(

  44. dennis said:

    it might not work for this file so handle that file manually ..
    than restart all ..

  45. Skinberry said:

    Thanks for responding, Dennis !

    I’ve got these files in /ffp/lib as you said. But how do i use them ?
    Is there a script to modify to point to these files ?
    Or do i have to copy them to a location ?

  46. Dennis said:

    if you have the files it should work

    please use latest version at nzbget.sourceforge.net

  47. Skinberry said:

    Dennis,

    Ik heb nogmaals gekeken of de libstdcc files aanwezig zijn en ja…dat zijn ze !
    Maar wat ik ook doe (rechten veranderen bijv.), ik krijg nzbget niet aan de praat.
    Ik gebruik versie 0.5.1 van deze site. Die zou toch moeten werken ?

    Heb je nog tips ? Of iemand anders ? Moet nzbget deze files ergens kunnen “zien” via een script ? Heb de CH3SNAS al meerdere keren gereboot…

    Please help !

  48. dennis said:

    ik gebruik inmiddels de 0.6.x versie

  49. Skinberry said:

    Ik geef de moed op….

    nzbget-0.6.0-testing-r252-bin-dns323-arm.tar.gz gedownload. Probeer dat te installeren (in /ffp/bin/), maar daarna gebeurt er niets meer.

    Linux zou in principe een leuk alternatief voor windows kunnen zijn. Alleen dan moeten de programmeurs iets meer moeite doen om te beschrijven wat en hoe het moet. Voor een leek is er geen beginnen aan. Steeds is er wel ergens een file-tje wat mist of net niet past.

    Fonz funplug is een voorbeeld hoe het ook kan. Dat gaat wel probleemloos.

  50. dennis said:

    Ik weet niet wat er bij jouw mis gaat ..

    ik heb het geinstalleerd. ik heb de nzbget.conf in /ffp/etc staan
    en de nzbget in /ffp/bin

    heb je in het forum van nzbget al gevraagd?

Leave a Reply

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

After submitting a comment you can MODIFY it again by clicking the message or name

*
To prove you're a person (not a spam script), type the security word shown in the picture. Click on the picture to hear an audio file of the word.
Click to hear an audio file of the anti-spam word