home network and backups: NAS convenience & simplicity

Merry Christmas! My gift to you is sharing some home networking and backup information!

The Dream:

...will start with a dream. This is... the Cadillac of home storage. The Drobo with SATA option. Considered it, but didn't go this route for our home network due to price. But you should check it out:

http://www.drobo.com/products/drobo-s.php

SATA, hot swappable, plenty of space for the next decade, and the way RAID is implemented for you makes it basically bulletproof for all your images, movies, music, software assets, etc. Even if 2 drives fail at once, your data is secure. That level of reliability comes with a high price tag, about $2500 ($800 for the chassis, $350 per each 2TB SATA drive). Overkill, but pretty awesome.

A More Reasonably priced setup:

Settling for less bulletproof, but with an affordable price and convenience... price tag only is about $119 per TB - a big discount to DroboS greatness.

Figure 1: Back of the Envelope design layout

The following outlines this configuration and setup for the home:

- NAS1 - 1 TB network attached storage (NAS) attached to your wireless router or Airport Extreme Base Station (AEBS) - (chose these MiniMax Hard Drives: same form factor as AirportExtreme, USB or Firewire, relatively quiet, $119 each via Amazon)

- NAS2 - another identical 1 TB drive daisy chained via USB

- you can configure AEBS (and other routers) to let drives attached to it to be shared drives between computers. If you have access to the wireless 802.11 network, you get access to the drives. In addition can password protect the drives themselves (beyond the 802.11 authentication), or require user authentication if you wish (easily setup).

- each Mac is set up to use NAS1 as it's source for music, movies, etc. -- can configure this in iTunes. Move your movies, and music to the spot on the NAS, and tell iTunes this is its 'home' for purchases, etc. -- this way you do not have to store 60, 80, 100 GB or however much music + movies + images on your laptop itself, but it's on the shared network drive. It has another added benefit if you're using any devices which consume iTunes media (E.g. aTV), in that these devices will effectively sync through the NAS through iTunes on your computer. If you're planning on using these media files for awhile you'll want them on expandable drive, instead of laptop harddrives which will fill up. (requires less management/time to organize)

- buffer size: any device that uses iTunes, or your computer in a client-server relationship will then be set to sync with the shared drive, not at your hard drive. There's a little added latency, but not much if you're on 802.11n, and you can set the buffer size on iTunes to "large" which means that you will download more of the song or movie first before playing it (less stuttering during playback)

- as of Snow Leopard's release, you can use network attached storage as TimeMachine backup destinations. Therefore, configured NAS2 to be the backup for both laptops.

- only extra item is that TimeMachine unfortunately does not allow you to backup from one NAS to another, only works from your computer's harddrive to the TimeMachine destination drive. This is something hopefully Apple will address in future releases. We want to back up from NAS1 to NAS2 and make sure that any shared drives on NAS1 also get backed up in case of failure. One option for dealing with this: write an rsync shell script that runs via cron, this is one way:

http://holtsblog.blogspot.com/2008/01/grok-rsync-for-automatic-redundant.html

Summary:

- movies, photos, images on a NAS to prevent you from using up space on laptop hard drive

- configure iTunes to use the NAS & therefore media through shared drive

- automatic backups from laptops to NAS2 via TimeMachine

- automatic scheduled backups from NAS1 to NAS2 via rsync script running on cron

...once setup, makes managing your digital assets less time consuming or worrisome.

Here are some links I found useful, if you're interested in going down this path:

- AEBS sharing drive:

http://www.apple.com/airportextreme/features/harddrivesharing.html

- TimeMachine backups allowed on AirPort Extreme:

http://www.macworld.com/article/132613/2008/03/airporttime.html

- Mac's synch-ing from a directory on NAS:

http://discussions.apple.com/thread.jspa?threadID=1368925

- iTunes over NAS via Airport disk:

http://www.tuaw.com/2007/03/02/tuaw-how-to-use-your-itunes-library-over-airport-disk/

- iTunes sharing libraries:

http://theappleblog.com/2009/09/11/itunes-9-at-home-with-home-sharing/

DNS hijack - high level explanation

http://www.theregister.co.uk/2009/12/18/dns_twitter_hijack/

A bunch of servers exist throughout the internet which map friendly human readable names (twitter.com, eyeonmajors.com) to IP addresses. These are DNS servers (Domain Name System servers) -- phonebook for the internet. When you type in "tigerwoods.com" to see his latest transgression admission, along the way DNS is queried and you are directed to the mapping to that IP address which presents the html and images of Tiger's statement.

A DNS hijack... is when the evil doers redirect the mapping, to a mapping of their own choosing. E.g. instead of twitter.com pointing to twitter's actual IP address, they point it to the IP of their own website where they can display whatever they want. (hey, maybe Tiger never admitted anything, and it is an ongoing DNS hijack?)

Once they have accomplished the hard part of hijacking DNS... it's very easy to create a landing page that looks identical in every way to the real site, but is in fact run by someone else. Look, Virginia, it even has the same URL. Looks legit, but is not. When people try to log in, the criminals setting up these sites can simply grab and store the username and password of the person attempting to login (phishing) for malicious uses.

So when/how can you be sure? HTTPS and SSL certificates can ensure that the site you are attempting to reach is the actual site. The site has been "notarized" in effect by an SSL cert, and while the evil doers can fake the look of a site and in some cases can even hijack the domain name (as they did with Twitter), they can't fake an SSL cert** -- it is bound to the domain name itself. (Not all sites use https and ssl certs)

...
http://twitter.com/coda/status/6789759675

Holts

** can't fake an SSL cert... unless the cert was encrypted using an MD5 hash, and the hacker had access to one or more PlayStation3's. ;)

lawn improvement

http://www.youtube.com/watch?v=OqHcm66s-kc

...since August 4th, decided to step it up. Watering, raking (a lot of dead grass), overseeding... took a picture every day. Hopefully in a month it will be all filled in. Plan to continue to log it with pictures every day or two, grow the slide show.

Better improvement in ~4-5 days than expected, it's filling in nicely. The 16 point sprinkler ($14.95) in the last two!

microformats ftw

Maybe not "cool" to say so, but MySpace++. Love the initiative to push forward with Microformats. µFormats ftw

http://www.readwriteweb.com/archives/myspace_to_bet_its_future_on_open_standards.php

Here's how MySpace's plan is unlike what Facebook is doing. The updates will be marked up for the types of activities they represent with standardized microformat code, beginning with the events format hCal and soon to include the book, movie or other review format hReview. Those little bits of code that will be added could have big consequences.

Keller says the company acknowledges that this won't be a small task for third-party developers, so in the meantime she is working on automated methods of pulling user data in from other sites' Application Programming Interfaces (APIs) and marking them up automatically, with the microformat code communicating what kind of updates they are (events, reviews, etc.)

programmable sprinkler

This is non-software related, but brilliant:
http://www99.epinions.com/review/Pulsating_Pattern_Master_Sprinkler/content_137910324868

Was hoping someone already invented something like this, as I have a non-circular lawn ;) Bought it. Very reasonable price, I hope it works well.

grok-ing POSH

..class names in XHTML are for semantics, not presentation. Yes, provide presentation with CSS calling out the class name, but the main purpose is for semantics.

http://www.w3.org/QA/Tips/goodclassnames

http://www.w3.org/2001/tag/doc/contentPresentation-26.html

Google acquires On2 (video compression)

http://googleblog.blogspot.com/2009/08/innovation-in-video-on-web.html

...high quality, searchable video programming on the web one day = inevitable. Surely Google wants to be at the epicenter.

App Store censors a dictionary?

This is pretty lame:

http://daringfireball.net/2009/08/ninjawords

In other words, not only must the dictionary be censored — a dictionary — but even after being purged of “objectionable” words it would only be considered with a 17+ rating. Even after agreeing to these terms, it took another two weeks for Ninjawords to appear in the App Store. According to Crosby, “We gave in and said fine, hoping that we could get on the App Store immediately since the solution to their rejection was a simple metadata change. However, the App Store reviewer would have none of that. We would have to resubmit an entirely new binary and get to the back of the queue before they would look at it again.”

Ajax API team leaves Google for VMWare

Exciting times.

VMWare is everywhere, so is cloud computing. For now I'll settle for being able to consistently build and deploy web apps on an elastic cloud (Heroku) in my spare time, but things are moving fast, which is fun to observe.

http://www.techcrunch.com/2009/07/13/google-loses-engineering-director-who-once-caused-steve-ballmer-to-melt-down/

Erlang -- let's get it on!

Messed around with Erlang for like the sum total of 2 hours over a year ago. But have decided recently that I'm going to sit down and little by little work towards learning it. It just seems like a great complement on the back end with Ruby, especially with items like CouchDB bubbling up. It's also fun and mind bending exercise to learn new syntax, along with a different language. Erlang is very different than imperative languages. Looking forward to the journey, while continuing to progress deeper into Ruby, JavaScript, REST, TDD, Rails...

So I'm working my way through this book: Programming Erlang: Software for a Concurrent World http://pragprog.com/titles/jaerlang/programming-erlang

mark_holton ~$ sudo port install erlang
---> Fetching gawk
---> Attempting to fetch gawk-3.1.6.tar.bz2 from http://mirrors.kernel.org/gnu/gawk
---> Verifying checksum(s) for gawk
---> Extracting gawk
---> Configuring gawk
---> Building gawk
---> Staging gawk into destroot
---> Installing gawk @3.1.6_0
---> Activating gawk @3.1.6_0
---> Cleaning gawk
---> Fetching tcl
---> Attempting to fetch tcl8.5.6-src.tar.gz from http://superb-west.dl.sourceforge.net/tcl
---> Verifying checksum(s) for tcl
---> Extracting tcl
---> Configuring tcl
---> Building tcl
---> Staging tcl into destroot
---> Installing tcl @8.5.6_0
---> Activating tcl @8.5.6_0
---> Cleaning tcl
---> Fetching tk
---> Attempting to fetch tk8.5.6-src.tar.gz from http://superb-west.dl.sourceforge.net/tcl
---> Verifying checksum(s) for tk
---> Extracting tk
---> Configuring tk
---> Building tk
---> Staging tk into destroot
---> Installing tk @8.5.6_0
---> Activating tk @8.5.6_0
---> Cleaning tk
---> Fetching erlang
---> Attempting to fetch otp_src_R12B-5.tar.gz from http://distfiles.macports.org/erlang
---> Attempting to fetch otp_doc_man_R12B-5.tar.gz from http://distfiles.macports.org/erlang
---> Attempting to fetch otp_doc_html_R12B-5.tar.gz from http://distfiles.macports.org/erlang
---> Verifying checksum(s) for erlang
---> Extracting erlang
---> Applying patches to erlang
---> Configuring erlang
---> Building erlang

internationalization

...since I'm going to have to speak Spanish in Alhambra, CA during the AT&T work stoppage... I might as well incorporate that into my web applications and "make lemonade" as the saying goes.

That's just a positive spin on having to go there. We coincidentally have homework due next week related to internationalization.

Been writing the Depot app via the PragProg book in Rails, but am doing so via Test Driven Development (slightly different from the book, per the awesome teaching of @tenderlove in the UWRails class -- this is the 2nd time I've taken this 3 month Rails class, and learning a ton more about Rails 2.2 and especially TDD)... am getting to the part where I'm incorporating internationalization.

I like this i18n features built into Rails, so I figured I'd blog about it, in case anyone else was interested: so certain items have translations. E.g. headers, titles, shopping carts, buttons, links -- not the data in the database, but all the labels and text that would help a user navigate through the application.

The translations are handled in a YAML file, which lets you call out the translation that matches it's hierarchy -- as shown below (Spanish, "es" shown here, there is an accompanying translation file for English, or "en"... could continue on as many languages as you wanted):


http://gist.github.com/66782

So the user sets their language preference via the dropdown (English is default, but they can choose whatever they want), then voila all labels that have a translation appear in that language selected! How does that work? Inside the code for all the pages, it just effectively says "give me whatever language is selected for layout.side.home, etc., and looks it up in the YAML file for that language above, grabs the matching value in that hierarchy, and out pops the word or phrase. For example, the markup for the sidebar links for "store", "questions", "news", "contact" look like this:

http://gist.github.com/66788

This is a common example, and almost identical to the example in the PragProg Agile Rails version 3 book -- maybe armed with this knowledge, I'll internationalize EyeOnMajors and Golfap as soon as it is riding on the Rails... :)

Giddyup.
Holts

Get Satisfaction -- customer dialogue

An example of what I'm using for EyeOnMajors to elicit customer feedback, ideas, etc.

http://getsatisfaction.com/eyeonmajors

forcing repaint DOM in IE6 and IE7?

I have loop that makes server-side Ajax calls on each iteration, and returns data. This could be 50 iterations, 500, 5000, whatever. Upon each iteration, I update that particular row with a server-side response (i.e. "success" or "fail"), and then update the total count processed ("26 out of 432 processed"). -- I update the innerHTML of a div or span, and another div showing the total number of records processed (at the end of an iteration, or in the onComplete callback handler of the Ajax.Request). That is all working perfectly fine... in Firefox, Safari, and every other browser except IE6 and IE7.

I have tried IE hacks such as adding a class to the div, and dropping a class from the div, adding and dropping child nodes from the div, etc -- to attempt to force IE to repaint, I have tried waiting a small interval, to no avail. If you pop up an alert("hai"); at the end of the iteration, the the IE6 and IE7 DOM will repaint with the updated values. (btw, how dumb is that?)

Does anybody have any tried and true ways to make IE6 or IE7 repaint a particular DOM element in these cases (without obviously popping an alert)? It works flawlessly in Firefox. I know IE is a steaming pile, but just wondering if anyone has had any success with a hack that works here.

Why "require 'rubygems'" In Your Library/App/Tests Is Wrong

I have been suitably convinced...



http://gist.github.com/54177

Recommended Ruby, Rails, REST books!

I have read the following books (some more closely than others). I'd recommend each of them for various reasons. All have been helpful in understanding many of the pragmatic concepts baked into Rails, understanding the Ruby programming language, and web development in general. I hope you have a chance to check them out on your own path of learning!
:Mark

RESTful Web Services

The Ruby Way, Second Edition: Solutions and Techniques in Ruby Programming (2nd Edition) (Addison-Wesley Professional Ruby Series)

The Rails Way (Addison-Wesley Professional Ruby Series)

Design Patterns in Ruby (Addison-Wesley Professional Ruby Series)

Ruby for Rails: Ruby Techniques for Rails Developers

Prototype and Scriptaculous in Action [Ajax]

Advanced Rails Recipes

Developing Facebook Platform Applications with Rails (Pragmatic Programmers)

Agile Web Development with Rails, 2nd Edition

'Daemon' by Daniel Suarez

Daemon, by Daniel Suarez. This looks like a really fun read, going to grab it! Check it out.

'carl_spackler' about to get ORM-ified

...yup, it's time... no more mysql gem... Should have done this earlier, but no better time than the present to implement...

...this weekend...going to convert any [current 'carl_spackler'] database queries that are mysql-specific, into ActiveRecord calls. This way, someone is just one adapter change away from using their database, any db they want that ActiveRecord supports, with Spackler. ...but really, it will make things easy for me to write to my db using the ActiveRecord syntactical sugar. Laziness... a virtue!

ActiveRecord::Base.establish_connection({
   :adapter => "sqlite",
   :dbfile => "db/mygolfdb.sqlite"
})

normalizing up 3 part names -- initial stake in ground

...all tests passing... ...collecting all 2008 PGATour data, and more Euro data now...

There are ZERO orphans in the 2008 PGATour data right now. Have collected each and every player's data for 36 tournaments in 2008. Including any other 3 part names.

The Player class is not in its ultimate form, but it is there and it splits names appropriately... still doesn't flatten special wacky characters and I'm not using any Bayesian techniques yet, but takes care of the 3 part names accurately: Jose Maria Olazabal, David Berganio Jr., Davis Love III, etc, etc.... also had to RegEx out of things like "Davis Love III (PB)"... the (PB) indicating the course name.

re = /\(\w{2}\)/
processed = name.gsub(re, "")

...re-scraping about 75 tournaments for PGA and Euro Tour with new names in the next 15 mins... pushed the new code to the carl_spackler GitHub repo .

CARL_SPACKLER::Player class:

vote for GitHub

Vote for GitHub as best bootstrapped startup in 2008. It takes 2 seconds and they are truly deserving:
http://crunchies2008.techcrunch.com/votes/?nominee_id=8&category_id=2