As readers of this less-than-often-updated blog are aware, I used my Raspberry Pi in conjunction with an old Linkysys WRT54GL wireless router into a wireless hotspot, complete with its own SMS gateway with the help of a 3G dongle. It worked flawlessly and I did receive emails and comments about how good the guide was (given the amount of detail and inate rambling, it better have been!). On top of that little bit of hardware, I also had an old Windows 7 laptop which was consolidating all the video cams in my home into a single web interface by means of a software called WebcamXP, which is a webcam monitoring tool. Its pretty neat in the way it allows users to specify URLs to their cameras for still shots and then strings them together to form a motion JPG or MJPG for a stream. The reason I used Windows 7 was because the software is purely Windows based. Linux has “mjpg-streamer“, which has two modes, a full video stream and also allows for still images to be taken at any point of time. Reloading the stills very rapidly would form an MJPG stream (like flipping pages of drawings to form a crude animation), pretty much the same thing WebcamXP does but I was too preoccupied with other stuff to actually write the script to reload the stills and also customize the webpages to give the same user interface. Why did I need the same interface? Well because the wife uses the system to view the cams as well and lets just say the less changes I have to instruct her to go through, the better it is for everyone. On top of this, I also had a simple file server running on the Windows laptop where I’d store cartoons and kid’s movies and stuff in lower quality MP4 format (separate from my very high quality mkvs in my DLNA NAS) for my son to stream to his iPad using the older AirVideo Server (AVS) software or played directly to the 55″ LG HDTV (yeah super cool, super big monitor) to which it was hooked up. The AirVideo Server, for all intent and purpose was also a Windows only software.
So the Pi was in my study, the laptop was in the TV console and I had two external USB drives hooked up to the laptop which served to store the cartoons and kid’s movies. Not a very pretty setup let me tell you (but tolerable since everything is hidden behide cabinet doors). Basically I had two very under powered devices, which served their purpose but the need to declutter eventually arose, so I had to look into consolidating the hotspot, webcam monitoring and the file server into a single device. Of course the unified machine would run Linux and be sufficiently powerful enough for the three tasks but also sufficiently powered for me to experiment and do projects on – something I couldn’t really do with the Pi. Don’t get me wrong, the Pi is a great little tool, but for heavy coding and projects, you just need something more powerful. I also needed something relatively small, that could fit inside the TV console, hidden from sight, with a sufficiently good BlueTooth range so I could use a wireless keyboard/trackpad to control it while lounging on the sofa 12 feet away.
After considering all the options, I decided to get an Intel NUC (Next Unit of Computing). I was gunning for the i3 version but the need to be some what future proofed steered me towards the i5 version. Now there are two i5 NUCs, one with a built in SATA connector and drive tray (D54250WYKH) for hooking up an internal 2.5″ disk drive (main disk is an mSATA SSD) and one without D54250WYK. The cost difference was actually quite significant, in the range of about S$200. On top of that I’d still need to get memory and a wireless combo card as the system comes bare. So with wanting to keep costs down, I decided to get the one without the internal SATA – after all I could still hook the USB drives to the many USB 3.0 ports the NUC had. I headed off to Sim Lim Square to see what kind of offers were up for grabs and heres where things get a little fortunate for me. On browsing the shops to compare prices, I came across one shop which had the D54250WYKH for the price of D54250WYK and they were also throwing in a Wireless N/BT combo card for free. I enquired twice on the price wanting to make sure the offer was right, and it was! I didn’t hesitate and got the system immediately with 4GB of memory (thing to note with the NUCs, you need _low_ powered memory – the system won’t even boot if you use regular memory). I later found out that the shop had misprinted the price of the item, but hey – all good for me!
So with the machine settled, I had to decide on the flavor of Linux to run and since I’d be coding and experimenting, I wanted an OS with a good package management system as I didn’t want to waste time on building dependencies for stuff that I needed. That meant almost immediately, that Slackware was out. Slackware is what I call the grand-daddy of the Linux flavors as using it teaches you about the inner workings of the Unix system. If you’ve grown up on Slackware, there isn’t a Unix system out there you can’t handle. Unfortunately, this familiarity comes from having to compile and build almost every single thing from scratch. It doesn’t have a good package management system because even if you do build Slackware packages, each of the dependencies is a separate secondary package which is not checked for when the primary package is being installed, unlike Debian’s apt-get or Red Hat’s yum. Slackware is a great teaching tool and its good hacker’s console, but its time consuming. Ubuntu is by far the most popular and easily installed (and maintained) of the Linux flavors and they had just released their very next LTS (Long Term Support) version, 14.04, so thats what I went with.
Moving the hotspot stuff with the SMS gateway was trivial, given it was essentially installing all the same packages I used on the Pi’s Debian image on Ubuntu – with a few changes to the PHP scripts and the Apache config (due to changes in PHP and Apache itself with the newer versions). All in all nothing really new from the instructions in my earlier posts, so I’m not going through any of that.
Setting up the fileserver was also trivial, as all it required was installing SAMBA and configuring it to read the 1TB SATA drive I put into the NUC (OS was on the mSATA SSD). Optimizing SAMBA for good read/write however was a different story. Using SAMBA with its default settings over my 500Mbps homeplug (which the NUC was connected to) I was getting like 2MB/s transfers (roughly 16Mbps). As you know, homeplugs never give you the speed at which they advertise, its a theoretical speed. You’ll be lucky to get half of what they advertise, more often than not its (at best) around a third due to interference and other factors. Adding in the overheads of the network protocol, disk write speeds and all, SAMBA should have been giving me maybe between 12 – 15MB/s. As such, 2MB/s was completely unacceptable. Thanks to the fact that we’ve been optimizing SAMBA at work for some of our backblaze like storage arrays, I had a rough idea what to tweak in the config. For those who don’t do the kind of work I do, you could also Google for optimizations (this is a good read). I made the following changes to the smb.conf:
read raw = Yes write raw = Yes strict locking = No socket options = TCP_NODELAY IPTOS_LOWDELAY SO_RCVBUF=131072 SO_SNDBUF=131072 min receivefile size = 16384 use sendfile = true aio read size = 16384 aio write size = 16384
Immediately I got a 5x boost in SAMBA read/write speeds to about 10MB/s – that’s about 80Mbps, which was at least tolerable if nothing else. That done, I needed to get AirVideo Server (AVS) running on the Linux. Fortunately the makers of AVS had created an java version that ran on Linux and I had actually packaged the whole thing (including all its dependencies, libraries, ffmpeg, etc) as a Slackware package for running on the Linux Live version of Slax. Turns out packaging everything was a good idea, since AVS jar file wasn’t compatible with the newer libraries and installing the old libraries would have definitely broken other parts of Ubuntu. So with the Slackware package I created I isolated all the old dependencies and libraries and the older version of java that would run the AVS jar into a separate directory and configured AVS to start only with those files. Pointed the config to the 1TB SATA drive where I transferred all the cartoons and kid’s movies and fired up the iPad to test and everything was running smooth.
So now, I only had the webcam monitoring system to deal with. Doing actual video streaming is extremely bandwidth intensive and viewing such streams over mobile data can deplete your data plan very quickly. Loading JPG stills rapidly one after the other (such that they appear like a continuous video feed, i.e MJPG streams) takes up significantly less bandwidth. So learning from the way WebcamXP did things, I would have to start the actual video streams locally (internal LAN) while setting up the MJPG stream such that it could be accessed externally (internet).
The “mjpg streamer” software I mentioned above would take care of creating the video streams and the stills without me having to write any code of my own. So I would run mjpg streamer for each camera on a different TCP port (eg: 81, 82, 83 …) using the following:
/usr/bin/mjpg_streamer -b -i "input_uvc.so -d /dev/video0 -f 30 -r 320x240" -o "output_http.so -w /var/www/htdocs -p 81"
Where /var/www/htdocs is any web accessible folder path where the stills are generated. With this, I could access “http://localhost:81/?action=snapshot” and view the still of the video stream at any point in time.
All that was required at this juncture, was to have a proxy like system where some script would take care of reloading the stills and to have a html page (an exact copy of the page WebcamXP uses so I don’t have to change the user interface) pointing to that proxy for the links to each of my cameras. Fortunately for me, real brilliant minds (unlike mine) always come up with these ideas before I do and there was a ready PHP camera proxy solution available. The core of the script is this:
$rand = rand(1000,9999);
$url = '<html link to mjpg streamer still>'.$rand;
$curl_handle=curl_init();
curl_setopt($curl_handle,CURLOPT_RETURNTRANSFER,1);
curl_setopt($curl_handle,CURLOPT_URL,$url);
curl_setopt($curl_handle, CURLOPT_USERPWD, "EXAMPLEUSER:EXAMPLEPASSWORD");
$buffer = curl_exec($curl_handle);
curl_close($curl_handle);
if (empty($buffer))
{
print "";
}
elseif($buffer == "Can not get image.")
{
print "Can not get image.";
}
else
{
header("Content-Type: image/jpeg");
print $buffer;
}
The “html link to mjpg streamer still” is simply:
http://localhost:81/?action=snapshot
That chunk of code repeats for each camera you have, with only the port number (ie:81) changing corresponding to the port numbers you used for each camera. The code just creates a random number appended to the end of the mjpg streamer snapshot link which forces the image reload and overwrite the old still image with a new one (via curl) to create the MJPG stream. From the code, you can see that one can also individually set passwords for access to the camera. The only downside of this script is that the usernames and password are in plain text, but thats still ok to a certain extent. Now assuming you have named the proxy script “camproxy.php”, to access the MJPG stream of each camera, you would call the following URL from your html page and it would show you the MJPG stream of your cam:
http://localhost/camproxy.php?camera=x (where x is the number of your camera 1, 2, 3…)
Cool stuff and the wife didn’t even know I changed anything!
And there you have it – all neat, all tidy and all consolidated, just the way I like things.
UPDATE 1:
AirVideo Server has a spanking new HD version which has an official Linux version – way to go InMethod!
UPDATE 2:
With my newer Sineoji 600Mbps AV2 homeplugs I didn’t see a significant increase in my SAMBA speeds, was more or less hovering around 80 – 90Mbps, but with my latest 1800Mbps AV2 gigabit homeplugs, I’m getting a cool 160 – 180Mbps with SAMBA.
