Jump to content
p0is0n

File Servers & NAS

Recommended Posts

15/10 EDIT: Feel free to discuss any kind of file server/NAS in this thread, lets not restrict it to simply HP microservers, although in my totally bias opinion they are great, there are plenty of other options in all shapes, sizes and flavours.

 

Firstly, I am not a very good writer, there are going to be typos, spelling errors (which I may pass off as typos), incorrect use of grammar and probably other mistakes. I also tend to repeat myself or go on and on about boring things, If this bothers you, consider stopping now. Secondly, this is part opinion, part experience and part review and possibly part tutorial, an overall impression I suppose.

 

I recently purchased a HP microserver to use as both a HTPC, a NAS & as an excuse to get my hands dirty in the world of Linux. I do have some basic Linux exposure, but I've never used it on a machine I owned and never learned the ins and outs. I have learned these are very popular pieces of kit, many people have them, many of those people have more than one and I know a few atomicans have quite recently acquired one. There are two flavours, the N40L (current model) and it's predecessor, the N36L. Both are very simillar, the N40L differing only in CPU clock speed and possibly a few other tweaks. These boxes are so full of potential, you can use them for many different things such as NAS, HTPC, VMWare Lab or (shock) even as a plain old server for small business/the home. I've been so impressed with it that I have decided to document my experiences with it so far and share them in the hopes of generating discusion and learning something new. I've read about many people using them for either NAS or HTPC but rarely both. My knowledge is pretty limited so if you feel you can suggest a better way of doing something, please do.

 

Before I even bought the N40L I was set on experimenting with ZFS as I've heard a lot of good things, and recently had some exposure to Nexenta at work, and I liked what I saw. As much as I would have liked to run a Solaris based OS like Solaris Express, OpenIndiana or Nexenta, I also intended to use the machine for a HTPC (read: needs to have a GUI) and so decided to use Ubuntu Linux as it is well supported and documented (good for a noob like me) and has a good range of software available. ZFS is not readily available under Linux due to licensing restrictions and needs to be ported over, I was sceptical at first but it seems the technology is quite stable and there are few reported issues.

 

First things first, a picture of the microserver in all its glory as I unboxed it earlier this week. While overall a very attractive, well designed box (it does fit a lot into a small chassis), I feel the top 1/3 could have been better utilised, although some people will feel compelled to install an optical drive here, I would rather cram more disks in. On the back of the door, you will find a small torx tool and as many screws as you will need, which was a nice touch IMO.

 

Posted Image

 

I won't bore you with pictures of the stock interiors, a quick google will fill you in, I will just get straight to it. I had 3x 2TB WD Green HDDs & 2x 1TB WD Green drives sitting in my desktop PC very close to full, this contains mostly music, tv shows, movies and other downloads. As I realised I was starting to accumulate a lot of data, the thought of losing a HDD became more of a concern. There is no mission critical data stored here, but losing 2TB of TV shows would take a while to recover from. I wanted some redundancy, and I wanted to be able to turn off my power hungry desktop when I was not using it. This is where the N40L begins to look attractive, using around 15W at idle and up to 40W under load I could leave it running 24/7 for torrents/NZB and serving media. The N40L can accomodate 4x 3.5" drives and 1x 5.25" drive by default, although I have seen as many as 9 HDDs stuffed into one of these boxes. It has 1x mini-SAS port (includes SFF8087 to 4x SATA cable), 1x SATA port (for ODD) and 1x eSATA port (on the rear), although by default the ODD SATA port and eSATA port do not run at full speed/AHCI mode. You will need a modded BIOS to unlock this feature, and a few other tweaks. Naturally, this was the first thing I did, as I intended on installing 5x 2TB 3.5" drives in total. I won't go into the specifics (unless anyone is interested) as it is pretty straightforward. After moving all my data off my 3x 2TB drives (took a few days coping to many smaller HDDs, mostly USB) I inserted 4x 2TB into the 4x non hot swap bays inside. I wanted to use the 5.25" bay to install my 5th HDD and so I happened to have a bracket available. It allows some ventillation which is a bonus, and is easy to slide in and out thanks to the design of the N40L. I ran the sata cable down the left side along with the SAS cable to the mainboard. For some reason the PSU only has molex connectors, so its BYO molex -> sata power adapter.

 

Posted Image

 

The N40L only has VGA out by default, and no audio out at all, so a low profile video card with HDMI would be required, I went with a HD6450 1GB for about $35 as it is low profile, passive cooled and has well documented compatability. Removing the motherboard to install either RAM or a video card is pretty simple, it does involve unplugging everything (about 7 cables, along the front of the board) and unscrewing 2 thumbscrews, then it slides out on a removable tray (a nice touch IMO). I found it easiest to remove the HDDs for this, so I could tuck the cables in there out of the way allowing the mainboard to slide in and out freely. Here is a picture of the board on its removable tray, with RAM & video card inserted.

 

Posted Image

 

As ZFS is a memory hungry file system, I decided that more than the 2GB of included ECC RAM would be required. By default the HP supports up to 8GB of RAM, but if you find the right sticks, it will support up to 16GB. It is extremely fussy with which sticks are compatible, there are a few unofficial compatability lists online but there also appears to be a few different versions of the mainboard, leading to hit & miss compatability. I went through 3 pairs before I found something 100% compatible with my particular server.

 

Firstly I tried KVR1333D3N9/8G (x2) which are half height Kingston sticks, non registered non ECC, they were $44 each. These only detected 8GB after many reboots. I tried one of these, plus the 2GB stick it came with and 10GB was detected, then I put an 8GB stick back in and 16GB WAS detected ONCE then upon next reboot it went back to 8GB. I tried about 20 reboots to see if there was a pattern and only saw the 16GB show up once, therefore I would not deem these compatible. Next I went for a safer bet, and grabbed a F3-1333C9D-16GAO kit, which are G-Skill Ares sticks, these were $84 for a kit of 2 or $42 each. These detected 16GB right off the bat, but subsequent restarts are again only detecting 8GB, and occationally as 16GB. These are supposed to be widely compatible with the N40L but not in my case. As the GSkill Ares stuff was no good for me for some reason, I decided to take it back to MSY and grab some Patriot G2 RAM which they also carry. They didn't mind me swapping it once I explained it was incompatible with my mobo and that it did work, and that I did know what I was doing. This time I got a PD000248-PGD316G1333ELK kit which is 2x8GB kit of Patriot G2 RAM. It had slightly more clearance than the Gskill ram above due to a lower heatspreader and is also documented as widely compatible. I have met great success with this RAM so far, after several days of use it continuously detects 16GB of RAM installed, as does my OS (Ubuntu Desktop x64). Mission accomplished. Here is a picture of the clearance on the RAM (standard height) although note I an no longer using the orange G.Skill Areas RAM, it was what was in there when I took these pictures. RAM with tall heatspreaders will need to be modified to fit. I feel this demonstrates the overall 'perfect fit' feel of the box.

 

Posted Image

 

After reconnecting everything and booting up, it was time to install an OS. I opted for Ubuntu 12.04 desktop edition as I needed a GUI for HTPC duties. I won't get into this, it is pretty straight forward. I installed onto a 16GB USB stick for now, I have ordered an eSATA to SATA cable off ebay for about $1.50 and am considering purchasing a small SSD when it arrives as the USB is a bit slow sometimes. Once the OS was installed, it was time to install ZFS. The process is actually very simple, just a few lines in terminal.

 

$ sudo add-apt-repository ppa:zfs-native/stable
$ sudo apt-get update
$ sudo apt-get install ubuntu-zfs

Easy. After that, it was time to create my zpool. I am opting for 5x 2TB drives as apparently RAIDZ performs best with a vdev size of 3, 5, 7 or 9. To do so takes just one line.

All the info regarding ZFS on Linux can be obtained here.

 

$ sudo zpool create -f -o ashift=12 storage raidz /dev/sda /dev/sdb /dev/sdc /dev/sdd /dev/sde

I then installed created a few folders, installed samba, shared them and started dumping files onto it. I am no expert on samba so won't document that here, suffice to say it was easy enough even for a total noob like me and I was up and running in minutes. That is pretty much where I am at for now, I installed VLC to watch things while I look into a better solution that can manage my files etc. I also need to get sickbeard up and running but I've been short on time due to a rent inspection next week. Maybe a job for the weekend if I get enough cleaning done :( Here is what I have at the moment, I've been happy with ease of use and performance. If things catastrophically fail, I am sure I will update this thread. For now, I am enjoying the security of ZFS having copied a chunk of data over so far, still have a fair bit to go though!

 

$ sudo zfs list
NAME				USED  AVAIL  REFER  MOUNTPOINT
storage			2.59T  4.54T   117G  /storage
storage/downloads   486G  4.54T   486G  /storage/downloads
storage/movies	  861G  4.54T   861G  /storage/movies
storage/music	   189G  4.54T   189G  /storage/music
storage/tv		  996G  4.54T   996G  /storage/tv

$ sudo zpool status -v
  pool: storage
 state: ONLINE
 scan: none requested
config:

		NAME		STATE	 READ WRITE CKSUM
		storage	 ONLINE	   0	 0	 0
		  raidz1-0  ONLINE	   0	 0	 0
			sda	 ONLINE	   0	 0	 0
			sdb	 ONLINE	   0	 0	 0
			sdc	 ONLINE	   0	 0	 0
			sdd	 ONLINE	   0	 0	 0
			sde	 ONLINE	   0	 0	 0

Posted Image

 

Posted Image

 

That is all for now, I may write a part 2 if there is any interest detailing additional configuration such as samba, sickbeard, sabnzbd etc.

 

As always, I welcome any thoughts/opinions/criticisms/suggestions. Sorry for the long post, thanks for reading it, well done if you made it this far. :)

 

p0is0n

Edited by Nich...
Updating the title~

Share this post


Link to post
Share on other sites

Great little write up bud, Good to see someone not using Windows server around here look forward to the next part. I've documented my setup when I built and configured my 20tb all in one whitebox but I dont know if atomic really has the audience for it.

Share this post


Link to post
Share on other sites

Am I reading it right that the ZFS pool is using slightly more than half of the space for replication ?

 

Mud3: no way to know if anyone will appreciate the write up if not posted.

Share this post


Link to post
Share on other sites

Great little write up bud, Good to see someone not using Windows server around here look forward to the next part. I've documented my setup when I built and configured my 20tb all in one whitebox but I dont know if atomic really has the audience for it.

Thanks mate, I think I will include samba setup and some other basic stuff in part 2 when I get some time to write it up and make sure I am giving out the correct info.

I am still learning most of this as I go along, hence my hesitation to include some info as it might be wrong :)

 

I have been a windows man most of my life since most games only work on windows really, I am enjoying the change to linux so far and have learnt a lot already.

I may reinstall with ubuntu server instead as the desktop version is a bit bloated with crap.

 

Am I reading it right that the ZFS pool is using slightly more than half of the space for replication ?

 

Mud3: no way to know if anyone will appreciate the write up if not posted.

I agree with this, mudg3 I would read it, and I also might learn a few things :) Post it anyway IMO.

 

I have 5x2TB in RAIDZ which is equivalent to RAID5 basically, so total usable capacity approx 1.8TB / drive, with 1x drive used for parity.

$ sudo zfs list
NAME				USED  AVAIL  REFER  MOUNTPOINT
storage			2.59T  4.54T   117G  /storage
This shows 2.59TB used, and 4.54TB still free. I believe a small amount of space is also used for ZIL but im not 100%. I have basically only lost 1 drive worth of storage space in total.

 

Once thing I have noticed is that I am using 10+GB of RAM most of the time, this could be related to not having a swap file as well though, but I am not regretting the effort I put into getting 16GB to work.

Edited by p0is0n

Share this post


Link to post
Share on other sites

Nice post Poison, i like learning new things, and your post has inspired a few more google searches ;)

 

Good on ya mate.

Share this post


Link to post
Share on other sites

Good write up, poison.

 

All i did with mine for the time being is put win7 on it to use as a media and file/photo storage unit. Ive got 1x 1TB hdd and 1x 3TB hdd and the 250gig that came with it.

eventually i might make a raid once mechanical drives come down in price more. got other things to do with my money at the moment.

I did put in a gigabyte hd6450 card in it for the HDMI tho.

You should check out shopping express, they have an SSD sale on for the next few days.

Share this post


Link to post
Share on other sites

Thanks for he explanation. I'm also pretty new to Linux, I was doing a Certificate 3 in System Administratioon in the first half of the year and we were doing Redhat Linux at the time. I understand a few concepts but I still am uh more comfortable with Windows. Reading over the first post it make much more sense regarding storage size now.

 

 

I also had a shot at Freenas and ZFS pools last year with my main storage array, had a few hiccups though so put it aside. I think the next time I look at storage I'm going to do a test array and then move my current storage to backup status when I feel comfortable enough with Linux/Solaris/wherever I end up.

Share this post


Link to post
Share on other sites

Thanks for he explanation. I'm also pretty new to Linux, I was doing a Certificate 3 in System Administratioon in the first half of the year and we were doing Redhat Linux at the time. I understand a few concepts but I still am uh more comfortable with Windows. Reading over the first post it make much more sense regarding storage size now.

 

 

I also had a shot at Freenas and ZFS pools last year with my main storage array, had a few hiccups though so put it aside. I think the next time I look at storage I'm going to do a test array and then move my current storage to backup status when I feel comfortable enough with Linux/Solaris/wherever I end up.

Cool, good luck with it and feel free to ask any questions. I've done a fair bit of research on ZFS and recently done my nexenta certification (CNA & CNSP ) which is based on Open Solaris, but I didn't want to use it for this box as I wanted a GUI too but for a pure storage box I think it's excellent. There is a free edition for up to 18TB capacity, any more and you need a license.

 

Nice post Poison, i like learning new things, and your post has inspired a few more google searches ;)

 

Good on ya mate.

Glad you enjoyed it :)

 

Good write up, poison.

 

All i did with mine for the time being is put win7 on it to use as a media and file/photo storage unit. Ive got 1x 1TB hdd and 1x 3TB hdd and the 250gig that came with it.

eventually i might make a raid once mechanical drives come down in price more. got other things to do with my money at the moment.

I did put in a gigabyte hd6450 card in it for the HDMI tho.

You should check out shopping express, they have an SSD sale on for the next few days.

One of the main reasons I wanted one of these micro servers was redundancy. I previously had 3x 2TB and 2x1TB hanging off my mobo and if any of them had died I'd have lost a lot of info. Now I can relax a little more, obviously I still have backups of my important data, but I don't have a lot of that these days. Glad you enjoyed the write up :) Feel free to share any updates on your own server in this thread too, that goes for anyone else who has one also.

 

 

As discussed above, I found that running Ubuntu off my USB was a bit too slow and wanted a more responsive GUI. For a command line only the USB is fine, but I think adding a GUI really kills it, and it doesn't handle the constant writing well. I decided to install my 250GB HDD that came with the Microserver to use for an OS, this means using the eSATA port as there are no other ports available, so I had to get an eSATA to SATA cable for this. I bought one for a few dollars on eBay on Wednesday and it arrived on Friday. I ran the cable from the rear eSATA port through the gap above the expansion slots, up the side of the PSU (fit very nicely) and up to the HDD. I managed to fit it into the top above my 2TB drive and its held there with some tape for now.. I will need to order a 5.25" to double 3.5" bracket eventually, but am waiting till I need some other stuff as postage to WA is dealbreaker for small orders. Here is a picture of the 6th HDD jammed in for now. The lid still fits on and clears the top HDD by a few mm. Lucky it is a slim 3.5" drive.

 

The 1M eSATA cable is a bit too long, I also ordered a few 0.5M cables from ebay for about $1.50 each, but they will take a few weeks to ship from yumcha in asia somewhere.

 

Posted Image

 

Posted Image

 

This created my first hiccup. When I created my vdev I used /dev/sda to /dev/sde and when I added the extra HDD it took the position of /dev/sde and pushed my 2TB t o/dev/sdf so this took 1 disk out of the vdev. It turns out it is a simple solution to fix it, the more I learn about ZFS the more I like it's simplicity. Firstly I needed to export my zpool.

 

$ sudo zpool export storage

Done. Now I need to import the zpool again, but this time I am going to add a command to import by disk-ID rather than device as it appears that wasn't the smartest choice when first setting it up.

 

$ sudo zpool import -d /dev/disk/by-id/ storage

Sometimes it is not always possible to export the zpool, such as if the HDD with the OS dies, or my thumbstick dies or something. In this case, it is still possible to recover the zpool with no significant effort. You can simply run the import and use -f (to force it) if it doesn't pick things up.

 

I then ran a scrub to make sure nothing bad happened, I wasn't expecting it, but it pays to be safe. No errors detected anyway :)

 

$ sudo zpool status
pool: storage
state: ONLINE
scan: scrub repaired 0 in 4h38m with 0 errors on Sat Sep 15 07:36:40 2012
config:

		NAME										  STATE	 READ WRITE CKSUM
		storage									   ONLINE	   0	 0	 0
		  raidz1-0									ONLINE	   0	 0	 0
			ata-WDC_WD20EARX-00PASB0_WD-WCAZAE889112  ONLINE	   0	 0	 0
			ata-WDC_WD20EARX-00PASB0_WD-WMAZA8002131  ONLINE	   0	 0	 0
			ata-WDC_WD20EARX-00PASB0_WD-WCAZAE830003  ONLINE	   0	 0	 0
			ata-WDC_WD20EARX-00PASB0_WD-WCAZA9329729  ONLINE	   0	 0	 0
			ata-WDC_WD20EARX-00PASB0_WD-WCAZAE689772  ONLINE	   0	 0	 0

errors: No known data errors

I didn't actually get a chance to install a new OS over the weekend but have no issues with things running how they are for now. My intention is to install ubuntu server, as I believe the kernel is different and more suited to this kind of work, and a more lightweight GUI such as LXDE. I haven't decided yet though.

 

That is all for now, just a quick update on things :) I am still working on a real part 2 with more details on configuration such as samba etc.

Edited by p0is0n

Share this post


Link to post
Share on other sites

What kind of speeds are you getting on the array bud?

I'm not sure if this is the best way to check or not, but it seems to give realistic information.

 

# time dd if=/dev/zero of=/storage/ddtest bs=1024000 count=20000
20000+0 records in
20000+0 records out
20480000000 bytes (20 GB) copied, 211.839 s, 96.7 MB/s

real	3m31.842s
user	0m0.076s
sys	 0m26.438s

So approx 97MB/s write speeds.

 

# time dd if=/storage/ddtest of=/dev/null bs=1024000

20000+0 records in
20000+0 records out
20480000000 bytes (20 GB) copied, 70.278 s, 291 MB/s

real	1m10.281s
user	0m0.044s
sys	 0m15.765s

And just under 300MB/s read speeds. I am pretty happy with the performance and it could only get better adding an SSD for cache, or a second vdev as I believe ZFS can read/write to multiple vdevs similar to striping.

 

I've also been keeping an eye on the HDD temps since sticking in the 250GB drive in last week, as I have gone outside the standard use by placing 2x 3.5" drives in the top section, where there isn't much in the way of airflow.

So far doesn't seem like it will be an issue, but will be keeping tabs. The top 2 drives are a few degrees warmer than the others even when not in use.

 

# hddtemp /dev/sd[abcdefg]
/dev/sda: WDC WD20EARX-00PASB0: 29°C
/dev/sdb: WDC WD20EARX-00PASB0: 28°C
/dev/sdc: WDC WD20EARX-00PASB0: 28°C
/dev/sdd: WDC WD20EARX-00PASB0: 27°C
/dev/sde: VB0250EAVER: 31°C
/dev/sdf: WDC WD20EARX-00PASB0: 30°C
/dev/sdg: Patriot Memory: S.M.A.R.T. not available

Not sure why I'm getting those weird characters in the output, but I believe the info is accurate.

Share this post


Link to post
Share on other sites

Great little write up bud, Good to see someone not using Windows server around here look forward to the next part. I've documented my setup when I built and configured my 20tb all in one whitebox but I dont know if atomic really has the audience for it.

Beginning to see what you mean by this.. not sure anyone here really interested enough. I've written up a bit of a part two but it's not finished yet.. not sure there is really any interest.

 

I've had the box running sab+sickbeard+couch potato+headphones and hammering my connection the last few weeks. Also installed Webmin for ease of use. Haven't needed to plug in a screen or reboot the server for weeks. Starting to get the hang of this. This is what my part 2 is mostly about, but I don't think I've even seen any of these programs mentioned on atomic before so not sure anyone is that interested.

 

I tried to install ubuntu server edition on the 250gb drive but kept failing to copy the grub data to /boot and I don't know what the issue is, tried it a few times, then just went back to my USB. It's honestly working beautifully it's just a but sluggish from the GUI sometimes. My 50cm eSATA to SATA cable also arrived from eBay but am considering just taking the 250GB out as I could do without the extra power and heat right now, or using it as a download drive so I am not constantly downloading to the zpool, just copying completed downloads across.

 

I still haven't gotten around to ordering a real double bracket for the 5.25" bay but if I ditch the 250GB drive I could get a 60GB SSD. I wonder if it would be possible to use 20GB for OS and 40GB for ZIL or if the entire drive would need to be dedicated to cache. Anyone know about this?

Share this post


Link to post
Share on other sites

Theres a few who will read it. I just know that what I've done will go way over the head of a lot of people here. If your going to post it on OCAU (which you did) you may aswell post it here.

Share this post


Link to post
Share on other sites

Yea I've posted a few bits and pieces there, might finish what I was going to write on the weekend and stick it in here then :) Even if a few people read it and learn something that would be worth it.

Share this post


Link to post
Share on other sites

Yea I've posted a few bits and pieces there, might finish what I was going to write on the weekend and stick it in here then :) Even if a few people read it and learn something that would be worth it.

 

Yep anything to get them away from windows server....

Share this post


Link to post
Share on other sites

Personally, I'd LOVE a write up involving the above programs you've mentioned.

 

I've been wanting to setup such a system for well over a year now, but a lot of factors (financal being not the least of them )have prevented me from doing so. A Proliant build would be 100% perfect for want I want to acheive.

Share this post


Link to post
Share on other sites

Figured I should stop lurking and post - I'm always interested in this kind of stuff. My N40L is at the post-office now for pick up, replacing my massive Norco chassis-based NAS.

 

Keep the posts coming!

Share this post


Link to post
Share on other sites

Ok guys, glad to see a few of you are interested, gives me more motivation to write. Here is the first little bit so far, I will add another post for the rest when I finish writing it up :) If you have any questions feel free to ask, or if you notice any errors, please do let me know, I am still fairly new to linux.

 

To begin with, I am going to assume you have read the first part of this thread, or, have a fresh installed copy of ubuntu installed (or any version of linux really). I believe most of these applications are available under windows in some form, but I won’t be covering that.

 

What I will be covering is the basics of a NAS/download server and possibly a few more features later. This includes Samba, for file sharing, SabNZBD for downloading, SickBeard for managing TV shows, Couch Potato for movies and Headphones for music. These programs will help to manage your media through a web interface, and can perform automatic downloading for you. I find SickBeard the most useful, I simple add a TV show to my library, and I can start downloading any episode or season I want. It will search for it and queue it up in Sab based on quality I select, when completed it will then be moved to the appropriate folder in my storage pool. It can scan for and detect the episodes I already have, and can rename them appropriately. It can also download new episodes automatically as they are released. Couch Potato and headphones perform similar functions, but for movie and music respectively.

 

Disclaimer: Before I get started I just want to point out that there are loads of guides for how to do the below out there, some much better explained than I ever could. I am still a linux noob but quickly getting my head around things. The information below may not be 100% accurate, or there may be better ways to do things. This works for me though, it should work for you too.

 

SSH

 

First thing to do is install Open SSH so we can SSH to the machine. This is extremely simple. Just type in the commands below or copy and paste.

sudo apt-get update
sudo apt-get install openssh-server
Done. Now you can open putty or terminal or whatever you want to use and connect to the machine with its hostname or IP and a valid user account.

 

Samba

 

Now you are ready to install Samba so you can share your media collection. After all, isn’t that what most people use a NAS for? My server currently contains approx 3TB of music, tv and movies and since I set it up, has been downloading around 10-12GB a day of new content for me automatically. My samba config is very simple, since there is only me and my girlfriend and occasional guests at my house, there isn’t much need for security. You would need to be connected to my LAN before you could see the server anyway.

 

To install samba just type the following into terminal and follow the prompts, basically, press ‘y’ when it asks for confirmation.

sudo apt-get update
sudo apt-get install samba smbfs

Then to configure samba, we need to edit the config file. I prefer nano over VI but if you’re using a GUI you can also type ‘gedit’ which stands for GUI text editor and you will see something similar to notepad in Windows. Here is the path to the file too.

sudo nano /etc/samba/smb.conf

You will see something like this, the first thing to do is change the workgroup to whatever you use for Windows, I just use WORKGROUP for this example as it is the default.

[global]
## Browsing/Identification ###

# Change this to the workgroup/NT-domain name your Samba server will part of

   workgroup = WORKGROUP
Next up you want to check for valid credentials when someone wants to connect. By enabling this option you will need to have a valid user account on the server. I just use the one account for everything, like I said it’s not a super locked down system protecting multiple users private files or anything, so this will do.

 

Uncomment the line ‘security = user’ by removing the # or ; infront of it, from my observation # is a comment and ; is a setting that’s not in use. Also add the second line as below, this tells samba to consult this location for a list of users.

####### Authentication #######

# "security = user" is always a good idea. This will require a Unix account
# in this server for every user accessing the server. See
# /usr/share/doc/samba-doc/htmldocs/Samba3-HOWTO/ServerType.html
# in the samba-doc package for details.

   security = user
   username map = /etc/samba/smbusers
Now there is one last thing to change, we need to specify which files to make available. If you’re reading through the config file, you will see the option to share home folders, but I prefer not to enable this, as it’s not going to be used as a daily machine slowly acquiring files I might want later, so the home folder will largely remain empty.

 

Scroll to the bottom of the config file and create something like this, there is unlikely to be anything similar there so just add shares as necessary. I am only sharing my storage volume, it contains subfolders such as tv, movies etc. You could share these individually if you like.

#
# My Shares
#
   [storage]
   comment = Storage
   browsable = yes
   read only = no
   path = /storage

 

I found I had to set r+w+x for each of these folders or I had issues, I am not sure why that is, but again given low key implementation I didn’t see a problem. I am not sure if there would be one? Just remind anyone accessing samba that they shouldn’t delete any of your files ;) This is an area where ZFS snapshots could be a lifesaver. The following command should do it.

sudo chmod -R 777 /storage
Note: In retrospect this may have been because the files in my /storage folder are owned by root, I’ve come to learn a little more about linux permissions recently. As long as you are the owner of the folders/files you shouldn’t need to do this step. If it doesn’t work then give this a go.

 

Next we need to create a password. You can use your default user or create a new one. I used my default user. Type the following, replacing <username> with your linux user account. This will create a password for samba.

sudo smbpasswd -a <username>

Next you need to add the user to the smbusers list we referred to earlier. Open the following file in nano and add <username> = “<username>” eg. p0is0n = “p0is0n”. This completes mapping the samba account to a user account.

sudo nano /etc/samba/smbusers
p0is0n = "p0is0n"

You can create a new user first, then add that user to the config file if you prefer. It doesn’t matter too much IMO. You should now be able to navigate to the share from Windows. Hold Windows Key and hit R to bring up a run window then type \\<server name>\<share> and you should be prompted for your user account and password. You should also be able to copy files to this location, rename, modify, delete and otherwise have full access to your files.

 

SickBeard

 

Before installing sickbeard, we need to check if python is already installed. Just type ‘python -V’ in terminal. Check for 2.5, 2.6 or 2.7, I am using 2.7.3. If it is not present, install it, but if you’re using Ubuntu it should be.

 

There are a few other prerequisite programs, install as below.

sudo apt-get update
sudo apt-get install python-cheetah
sudo apt-get install git

If you don’t know, put simply, git will copy some code/files from the source to whatever destination you choose, it is quite common in linux. I am installing this in my home folder, you can do it anywhere though. Obviously <user> represents your user account.

cd /home/<user>
git clone git://github.com/midgetspy/Sick-Beard.git sickbeard

Now you want to change to the new directory, remembering linux directory names are case sensitive, and then use python to execute the program.

cd sickbeard
python SickBeard.py

That is really all there is to it, now just open a browser, type in the IP of your server and port 8181 eg. 192.168.1.100:8081 and you’ll be greeted by the sickbeard interface. It is fairly intuitive. I won’t get into config here yet, as you need to install sab too.

 

Sabnzbd

 

Before you can install sab, you will need to add the repository so ubuntu it can find the source. Enter the commands below to install sab.

sudo add-apt-repository ppa:jcfp/ppa
sudo apt-get update
sudo apt-get install sabnzbdplus

Once it is installed, it is a simple case of typing ‘sabnzbdplus’ to run the software. It runs as a daemon so it would be best to open a terminal on your server and run it there, as if you’re using something like putty closing the window will end the program.

 

Note: DO NOT type sudo in front of sab when you launch it, I accidentally did this once, everything it downloaded then belonged to root and my post processing scripts didn’t work and I had to change the owner and permissions for all the files I had downloaded for the last few days. Was a pain :)

 

To access sab just open your browser, type in the server IP and port 8080 eg. 192.168.1.100:8080. You can change the port for this or sickbeard in the config if you wish. If you are in front of the machine you can use localhost:8080.

 

You will be prompted with a setup wizard, just follow the prompts. You will need to have an account with a news server, I use astraweb as it is quite cheap, and you will also need an index site. I use NzbMatrix and Newzbin, I also think its worth the few dollars to go premium on NzbMatrix. You can add any other index site you wish, depending on the kind of content you are looking for. There might be some good ones out there for music, I haven’t looked into it much yet. I am still a member on a few private trackers (torrent) for less common music. Once you have entered all your details, restart the server, there is a button for this.

 

Now you can start sab and leave it running, i’ve had mine going for weeks and never needed to reboot it. I haven’t set it to start automatically yet, as it looks like it’s a little more involved than just a ticking a box or changing a 0 to a 1 somewhere.

 

Open your sickbeard interface <hostname>:8081, go to config and search settings, then fill in the details for sab here and enter a category name like ‘tv’ so you can use the category later. You can chose the naming conventions you like to use under post processing, but leave everything else disabled as we will use sab for post processing. Save and restart Sickbeard.

 

Couchpotato

 

Installing this is a very similar process to Sickbeard. First create a directory and then navigate to it, I called mine couchpotato. Copy the code, then run the program.

cd /home/<user>
git clone https://github.com/RuudBurger/CouchPotatoServer.git couchpotato
python /couchpotato/CouchPotato.py
The default port for couch potato is 5000. Once installed, you’ll want to go into settings and enter the details for sab as the downloader and whichever index sites you use as the searchers. I also changed the port to 8082 to keep things simple. Also set category as ‘cp’ or ‘movie’ or something.

 

Headphones

 

Pretty much as above again. Starting to get the hang of it by now I hope :)

cd /home/<user>
git clone https://github.com/rembo10/headphones.git headphones
python /headphones/HeadPhones.py
Headphones will operate on port 8181, i changed this to 8083 to keep the pattern going. Set category as ‘music’.

 

Additional Config

 

I would suggest now taking the time to slowly begin adding media to these programs. Files will need to be named a certain way before they are recognised, for example, before sickbeard will recognise an episode of a tv show, it will need to contain the name of the show, or be in a folder of that name, and contain both the season number and the episode number eg. Tv.Show.S01.E01.Name or TV Show/S01E01 or TV Show/01x01 - Name etc. It needs enough info to figure out what it is looking at, and refer to it’s database. Same with couch potato (needs the year of the movie and the title) and headphones (needs artist, album and year). There is plenty of info on the net about using these programs so I won’t go into much detail, they are all fairly intuitive. A quick note on sickbeard though, make sure you pause any shows you don’t want to automatically acquire new episodes for, and you can do a mass update for all shows for things like quality instead of doing it individually as you add them. I spent many many hours adding shows and renaming episodes but now my media is more organized than ever.

 

Now that the basics are out of the way, you should be able to manually add something to any of these programs and it will search for it, start downloading in sab and remain in the sab download folder. This might be okay for some, but we can do better. If it is a little obscure, it may not be found straight away, I’ve come across a few things I couldn’t find, but 99% are available. This may be because I need a better index or news provider, not sure. You can still use torrents for those things, which i’ve done a couple of times :)

 

Lastly, I want to cover post processing. This is the ability to automatically move, rename and sort downloads into appropriate folders. I’ve only set this up for sickbeard so far as the others don’t include a script for this, but I’ve discovered there are plenty others are sharing, and you can make your own.

 

First thing to do is copy the contents of /sickbeard/autoProcessTV to a folder of your choice. I made a folder in my home directory called scripts. Rename autoProcessTV.cfg.sample to just autoProcessTV.cfg and fill in the blanks eg. host, port, username, and password then save. In sab go to settings, then folders, and enter the path to your scripts folder, now sab will be able to find the script. Now go to categories, create a new category for ‘tv’ which would be the label sickbeard is sending to sab. Change processing to ‘+delete’ and script to ‘sabToSickBeard.py’ and save. Now when sab completes a download tagged as tv, it will run this script which will rename the files, move them to the correct location, specified when you added the show originally. It will then clean up the original files (delete from downloads directory). I was a bit skeptical about this at first, but it has worked flawlessly thus far :) I believe the process would be very similar for any other scripts, but as I said before, I haven’t looked very hard for ones for couch potato or headphones yet, I’ve just moved the few files I have downloaded so far manually.

 

Webmin

 

One last program that may be of value is Webmin, it basically simplifies some of the basic admin tasks you might do in front of your computer. Think of it like a kind of control panel, but in a browser. I will admit I haven’t used this much, but I did use it to do a few software updates and it is handy for keeping an eye on things like CPU/memory/disk usage and running processes. I am sure it has some real beneficial features but I’ve not needed to use it much yet, I’ve mostly been using terminal.

 

There are a few dependencies it relies on so we will need to install a few programs. If it is already installed, it won’t install twice so don’t worry about that.

sudo apt-get update
sudo apt-get install perl libnet-ssleay-perl openssl libauthen-pam-perl libpam-runtime libio-pty-perl apt-show-versions python

Next download the webmin package and then install it. It will download to whatever directory you’re in at the time so might be worth changing to your home folder.

wget http://prdownloads.sourceforge.net/webadmin/webmin_1.580_all.deb
sudo dpkg -i webmin_1.580_all.deb

Now it is available to access. Webmin uses SSL so you will need to use https: when contacting it. It runs on port 10000 by default eg. https://hostname:10000 then login with your linux username and password. You will see something like this when you’re in.

 

Posted Image

 

That’s about it from me now, I hope some of you found this at least a little helpful, I know it’s not a very complete guide and doesn’t get in too much detail. If you have any questions please ask, I wrote most of this from memory which goes back a few weeks (my memory is terrible I smoked way too much weed for a few years haha) so I may have forgotten some important steps. I would also appreciate any feedback or any suggestions or criticisms. If I could improve I would like to try, and like I said at the start I am a bit of a linux noob so if you see a mistake or something that could be done better please let me know.

 

p0is0n

 

EDIT: Added the rest of what I've written. Cheers.

Edited by p0is0n

Share this post


Link to post
Share on other sites

Aha, something about Linux I know about !

 

If you want to change up the group ownership for the storage folder , login as root or use sudo and type :

 

chgrp <groupname> /storage

 

This way if in the future you do want to add more users, you can add users to the group you specified. You can leave root as the owner, and you should be able to set permissions on the /storage and sub-folders as rwx-rx-rx. This also means you don't need to worry about Samba users deleting files.

 

Oh yeah, if you do it this way you'll need to make sure that any of your programs that save to the /storage folder have appropriate permissions to write to that folder.

Edited by Devilsmurf

Share this post


Link to post
Share on other sites

Thanks mate, permissions is something I am still getting my head around. I've been using chmod -R user:group <target> so far as I didn't know about chgrp so will check it out.

I have it set the way it is now as I generally manage everything from my PC so I can move/copy something and delete the remnants hassle free if necessary. I'd be more concerned if there were other users though, much more concerned ;)

Edited by p0is0n

Share this post


Link to post
Share on other sites

For your situation, your only concern I'd say would be accidental deletion. Group permissions really come into their own when managing multiple users.

 

One thing to remember about permissions, you can set them on both files and folders. Folder permissions override file permissions, so if you have 777 on a file and move it to a folder with 700 and try to access that file with anything other than root, you won't be able to.

 

While I'm posting, in the future would your prefer to read permissions as rwx-rwx-rwx or as 777 ? I'll stick to one convention so I don't inject confusion into the thread.

Share this post


Link to post
Share on other sites

Cheers Devilsmurf, this entire exercise has been all about learning for me, as I knew fuck all about linux before I started. I've been doing almost everything via SSH so it's forced me to learn and remember a few things.

I appreciate your explanations, filling any gaps in my knowledge is the goal. I am interested in doing things right, as much as making it work, so will look at amending my permissions and maybe turning this into a full guide some day.

 

777 is fine with me :) it's also easier to type.

 

EDIT: Post above updated with the rest of my part 2.

Edited by p0is0n

Share this post


Link to post
Share on other sites

One last thing I forgot to mention, this isn't just about my server and doesn't just have to be about HP Microservers.. I might see about getting the title changes to File Server Club or something, since not many people have N36L/N40L on here. There is a similar thread on OCAU (which I have been active in recently) which is hundreds of pages long and has over 7000 posts. I don't expect we will get that big here but lets not pigeonhole it too much.

 

Cheers

Share this post


Link to post
Share on other sites

just a +1 to watching this thread :)

 

I also have a hp microserver and for the past year or so it's been running WHS. While this is great from a windows backup perspective (i.e. backing up the home PCs), it falls down in a couple of major areas IMO - storage mgmt & serving media.

 

Because of this I'm changing my OS.

 

On a spare lappy I've been playing with ubuntu server and that will probably be the way I go. had a very quick play with mediatomb as a media server, seems ok if not basic but I'm still looking around.

 

As for samba, yes I got it working but I think there is a lot more to it than I've been playing with :)

 

Anyway, keep the posts up p0is0n

Share this post


Link to post
Share on other sites

One last thing I forgot to mention, this isn't just about my server and doesn't just have to be about HP Microservers.. I might see about getting the title changes to File Server Club or something, since not many people have N36L/N40L on here. There is a similar thread on OCAU (which I have been active in recently) which is hundreds of pages long and has over 7000 posts. I don't expect we will get that big here but lets not pigeonhole it too much.

 

Cheers

 

You have to remember though that the vast majority of those posts are just total crap. People posting un related content or just having stupid conversations the only reason that thread is still open and unlocked is because every 2nd thread in storage section would be (how do I install windowz on my n40l)

 

Quality over quantity my friend.

Share this post


Link to post
Share on other sites

Am pretty keen on getting one of these to replace my current NAS (Netgear ReadyNAS) is just too noisy.

 

Another great tip if you are using Linux based machines (and possibly BSD) is to install Avahi (usually installed by default on Ubuntu) and enjoy the goodness that is ZeroConf networking.

 

Adding a simple XML file into /etc/avahi/services will automatically publish the service to zero conf and any OS with zero conf support (primarily OSX although named Bonjour in Apple land) will discover all of your services. I use it to discover HTTP and SSH services without needing to screw about with static IP's etc.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×