I’ve got a big announcement today:
I will be doing more Deep Learning and Convolutional Neural Network tutorials on the PyImageSearch blog over the coming months.
I’m dead serious about this — and I’ve put my money where my mouth is and invested in some real hardware for deep learning.
To learn more about my investment, the NVIDIA DIGITS DevBox , and the new tutorials coming to the PyImageSearch blog, keep reading.
Hands-on with the NVIDIA DIGITS DevBox for Deep Learning
For anyone that is interested in the NVIDIA DIGITS DevBox for deep learning — and perhaps more importantly, the rational that led to me purchasing a pre-configured deep learning system instead of building my own — I’ve included my experience in working through the decision process, making the purchase, and unboxing the system. In future blog posts, I’ll be reviewing how I’ve setup and configured the system for my own optimal setup.
NVIDIA DIGITS DevBox specs
Let me tell you: the NVIDIA DIGITS DevBox is a beast.
In terms of system specs, the DevBox sports:
- 4 Titan X GPUs (12GB of memory per board )
- 64GB DDR4 RAM
- Asus X99-E WS motherboard
- Core i7-5930K 6 core processor operating at 3.5GHz.
- Three 3GB SATA 3.5″ hard drives configured in RAID5 (useful for storing massive datasets)
- 512GB SSD cache for RAID
- 250GB SATA internal SSD (where you’re store your system files, source code, and other “most accessed” files)
- 1600W power supply
But it’s just just the hardware that makes the NVIDIA DIGITS DevBox awesome. It also comes pre-configured with:
- Ubuntu 14.04
- NVIDIA drivers
- NVIDIA CUDA Toolkit 6.0-7.5
- cuDNN 4.0
- Caffe, Theano, Torch
This machine is no joke — and it’s not cheap either.
Weighing in at $15,000 , this isn’t your standard desktop machine — this system is intended for researchers, companies, and universities that are doing real work in deep learning.
So, you’re probably wondering…
“You’re not a deep learning researcher, Adrian. And while you own companies that involve computer vision + machine learning, why do you need such a beastly machine?”
Which brings me to the next section of this blog post…
Investing in the future of PyImageSearch, my companies, and myself
On the surface, you likely see two sides of Adrian Rosebrock:
- The blogger who writes weekly blog posts and email announcements.
- The wordsmith , teacher , and educator who as authored Practical Python and OpenCV as well as the PyImageSearch Gurus course .
But there is also a third side that doesn’t often get discussed on the PyImageSearch blog (besides the occasional note here and there): the entrepreneur and (occasional) consultant.
It’s becoming increasingly rare that I can take on new consulting/contracting work, but when I do, I tend to be very selective about what the project and budget is. And over the past few years, I’ve noticed I’ve been using more and more deep learning within my projects (both for contracting work and for personal/business projects).
This may appear in stark contrast (and on the surface, perhaps a bit hypocritical) to my blog post on getting off the deep learning bandwagon — but the title of that article wasn’t the primary point.
Instead, the purpose of that (controversial) blog post was to drive a single detail home:
“Machine learning isn’t a tool. It’s a methodology with a rational thought process that is entirely dependent on the problem we are trying to solve. We shouldn’t blindly apply algorithms and see what sticks [aside from spot-checking]. We need to sit down, explore the feature space (both empirically and in terms of real-world implications), and then consider our best mode of action.”
Deep learning, just like Support Vector Machines, Random Forests, and other machine learning algorithms all have a rational process and assumptions to when we should use each particular model. There is a time and a place where we use deep learning — you just need to be mindful in your selection of algorithms for a particular program.
My hope in publishing deep learning tutorials on the PyImageSearch blog is to better illuminate when and where deep learning is appropriate for computer vision tasks.
So what does this have to do with the PyImageSearch blog?
I’ve said it before in other blog posts, and I’ll say it again here today:
What I love about running the PyImageSearch blog is writing tutorials related to what you , the reader, want to here about.
Every day I get more and more requests for deep learning tutorials. And up until 2 months ago, I was still writing the PyImageSearch Gurus course . I simply did not have the time, energy, or attention span to start planning out deep learning tutorials — which by their very definition take a lot more time and effort (in terms of thought process, computational effort, and experiments) for me to create.
Now that I’ve finished writing the PyImageSearch Gurus course, I’ve reclaimed a bit of my time.
But more importantly , I’ve reclaimed a bunch of my energy and attention — both of which are critical in creating high-quality tutorials. Over the years I spent in graduate school, writing my dissertation, running the PyImageSearch blog, authoring Practical Python and OpenCV , and creating the PyImageSearch Gurus course , I’ve mastered the ability to bang out 5,000+ words in a single sitting. Time isn’t a problem for me when it comes to writing — what really matters are my energy and attention levels.
Over the next year, you can expert more deep learning tutorials to be published on the PyImageSearch blog. It won’t be an immediate change , but it will slowly ramp up over the next 3-4 months as I start creating a backlog of posts.
The point is this:
If you’re interested in deep learning, specifically deep learning and computer vision, PyImageSearch is the place to be.
Running the numbers
Okay, so I’ve already mentioned that I invested $15,000 in an NVIDIA DIGITS DevBox — that’s not a small amount of money by any means.
So how did I justify this huge number?
As I mentioned above, I’m an entrepreneur, a scientist, and at heart, a business person — this implies there is (some sort of) logic behind my actions. If you show me the numbers, they work out, and they align with my goals — I can justify the investment and plan accordingly.
I started out the assessment by looking at the facts:
- Fact #1: I am currently spending either $0.65 per hour on an Amazon EC2 g2.2xlarge instance or $2.60 per hour on an g2.8xlarge instance. Most of my EC2 hours are spent using a g2.8xlarge instance.
- Fact #2: Both of these EC2 instances have less RAM and than the NVIDIA DIGITS DevBox. And based on the way Amazon computes virtual-CPUs , the g2.8xlarge has only marginally better CPU (again, if you can trust the vCPU allocation).
- Fact #3: Currently, EC2 GPU instances have only 4GB of memory with 1,536 cores (the Titan X has 12GB memory and 3,072 cores).
- Fact #4: The NVIDIA DIGITS DevBox has 4 Titan X GPUs (totaling 48GB of memory). The g2.8xlarge has 4 K520 GPUs (totaling 16GB of memory).
- Fact #5: 4GB alone just doesn’t cut it for larger datasets without spreading computaiton across multiple GPUs. That’s fine, but doesn’t seem worth it in the long run. Ideally, I can run either four experiments in paralell or spread the computation across four cards , thereby decreasing the time it takes to train a given model. The Titan X is clearly the winner here.
Next, I ran the numbers to determine the intersection between the hourly rates of an g2.8xlarge instance and the $15,000 upfront investment:
Figure 1:Plotting the intersection between the EC2 g2.8xlarge hourly rate and the upfront cost of $15,000 for the NVIDIA DIGITS DevBox. The break-even point is at approximately 240 days.
This gives me approximately 5,769 hours (~240 days) of compute time on an g2.8xlarge instance.
Note:That’s 23,076 hours (2.6 years) on a g2.2xlarge instance — again, I’ll reiterate the point that I’m mainly using g2.8xlarge instances).
With a break even point of only 240 days (where it can takes days to weeks for a single model to train), the decision started to become more clear.
Now, the next question I had to ask myself was:
“Do I order the hardware, put it together myself, and save money? Or do I go with a pre-configured machine pay a bit of markup?”
I’ll get a fair amount of negative feedback for this point, but in my opinion , I tend to lean towards “done for you solutions”.
Three reasons: I have limited time , energy , and attention .
Anytime I can spend money to pay a professional to outsource a task that I am either (1) not good at, (2) don’t like doing, or (3) is not worth my time/energy, I’ll tend to move the task off my plate — this is the exact rational that enables me to work smarter instead of harder .
So, let’s presume that I could buy the hardware to create a comparable system to the NVIDIA DIGITS DevBox for approximately $8,000 — that saves me $7,000 right?
Well, not so fast.
I’m not going to say what my hourly consulting rate is, but let’s (for the sake of this argument) say I charge $250 per hour of my time: $7,000 / 250 per hour = 28 hours.
In order for this time-to-money tradeoff (let alone the attention and energy it will take), within 28 hours of my own time, I need to:
- Research the hardware I need.
- Purchase it.
- Get it all together in my office.
- Assemble the machine.
- Install the OS, software, drivers, etc.
Can I do all this in 28 hours with minimum context switching?
Honestly, I don’t know. I probably could.
But what if I’m wrong?
And the better question to ask is:
What if something breaks?
I’m not a hardware person, I don’t enjoy working with hardware — it’s just the way I’ve always been.
If I build my own system, I’m my own support staff. But if I go with NVIDIA, I have the entire DevBox team to help support troubleshoot, and resolve the issue.
So, let’s say that it takes me a grand total of 15 hours to order the hardware, put it together, install the required components, and ensure that it’s working properly.
That leaves me with 28 – 15 = 13 hours of my time left to handle any troubleshooting issues that occur over the lifetime of the machine.
Is that realistic?
No, it’s not.
And from this perspective (i.e., my perspective), this investment makes sense. You may be in a totally different situation — but between the projects I’m currently working on, the companies I run, and the PyImageSearch blog, I’ll be utilizing deep learning a lot more in the coming future.
Factor in the fact that I not only value my time, but also by energy and attention , this further justifies the upfront cost. Plus, this better enables me to create awesome deep learning tutorials for the PyImageSearch blog.
Note:I’m not factoring in the increase of my utility bill which will happen, but in the long run, this becomes marginalized due to my ability to save time, be more efficient, and ship faster.
Ordering the NVIDIA DIGITS DevBox
Placing an order for the NVIDIA DIGITS DevBox isn’t as simple as opening a webpage, entering your credit card information, and clicking the “Checkout” button. Instead, I needed to contact NVIDIA and fill out the access form.
Within 48 hours I was in talks with a representative where I created the PO (Purchase Order) and shipment. Once the terms were agreed upon, I cut a check and overnighted it to NVIDIA. Overall, it was quite the painless process.
However, I will note that if you’re not doing a lot of shipping/receiving, the actual shipment of the DevBox may be a bit confusing. I personally don’t have much experience in logistics, but luckily, my father does .
I called him up for some clarity regarding “Freight terms” and “EX-WORKS”.
Essentially, the EX-WORKS comes down to:
EX-WORKS (EXW) is an international trade term that describes an agreement in which the seller is required to make goods ready for pickup at his or her own place of business. All other transportation costs and risks are assumed by the buyer. ( source )
What this boils down to is simple:
- NVIDIA will be putting together your system.
- But once it’s boxed up and on their loading bay, the responsibility is on you.
How did I handle this?
I used my FedEx business account and shelled out extra cash for insurance on the shipment. Not a big deal.
The only reason I included this section in the blog post is to help out others who are in a similar situation who may be unfamiliar with the terms.
Unboxing the NVIDIA DIGITS DevBox
The DevBox ships in a hefty ~55lb box, but the cardboard is extremely thick and well constructed:
Figure 2:The box the NVIDIA Digits DevBox ships in.
The DevBox is also very securely packed in a styrofoam container to prevent any damage to your machine.
While unboxing the DevBox may not be as exciting as opening a new Apple product for the first time, it’s still quite the pleasure. After unboxing, the NVIDIA DIGITS DevBox machine itself measures 18″ in height, 13″ in width, and 16.3″ in depth:
Figure 3:The NVIDIA DIGITS DevBox fully unboxed.
You’ll notice there are three hard drive slots on the front of the machine:
Figure 4:Three hard drive slots on the front of the DevBox.
This is where you slide in the three 3TB hard drives include in the shipment of your DevBox:
Figure 5:The DevBox ships with your 3TB hard drives. Luckily, you don’t need to purchase these separately.
Slotting the drives in couldn’t be easier:
Figure 6:Slotting the drives into their respective bays is easy. After slotting, the drives are securely locked in place.
On first boot, you need to connect a monitor, keyboard, and mouse to configure the machine:
Figure 7:Make sure you connect your monitor to the first graphics card in the system.
Pressing the power button starts the magic:
Figure 8:Booting the DevBox.
Next, you’ll see the boot sequence:
Figure 9:Going through the motions and booting the NVIDIA DIGITS DevBox.
After the machine finishes booting, you’ll need to configure Ubuntu (just like you would with a standard install). After setting your keyboard layout, timezone, username, password, etc., the rest of the configuration is automatically taken care of.
It’s also worth mentioning that the BIOS dashboard is quite informative and beautiful:
Figure 10:The BIOS dashboard on the NVIDIA DIGITS DevBox.
Overall, I’m quite pleased with the setup process. In under 30 minutes, I had the entire system setup and ready to go.
You’re not plugging that directly into the wall, are you?
Protect your investment — get a quality Uninterrupted Power Supply (UPS). I’ll detail exactly which UPS (and rack) I chose in the next blog post. The images gathered for this blog post were mainly for demonstration purposes during the initial unboxing.
In short, I would not recommend having your DevBox sitting directly on top of carpet or plugged into an outlet without a UPS behind it — that’s just asking for trouble.
Admittedly, this is a lengthy blog post, so if you’re looking for the TLDR, it’s actually quite simple:
- I’ll be doing a lot more deep learning tutorials on the PyImageSearch blog in the next year. It will start with a slow ramp-up over the next 3-4 months with more consistency building towards the enter of the year.
- In order to facilitate the creation of better deep learning tutorials (and for use with my own projects), I’ve put my money where my mouth is and invested in an NVIDIA DIGITS DevBox .
- The DevBox was a delight to setup…although there are a few practical tips and tricks that I’ll be sharing in next week’s blog post.
- If you’re interested in diving into the world of deep learning, the PyImageSearch blog is the place to be.
It’s worth noting that I already cover deep learning inside the PyImageSearch Gurus course , so if you’re interested in learning more about Neural Networks, Deep Belief Networks, and Convolutional Neural Networks, be sure to join the course.
Otherwise, I’ll be slowly increasing the frequency of deep learning tutorials here on the PyImageSearch blog.