mar 28

Netflix: We don’t need no stinkin’ data centers!

Generale, Hardware Upgrade, iPhone, Mobile No Comments »

Netflix, the streaming media giant, turned to Amazon for its cloud infrastructure because Amazon Web Services was — and remains — the biggest, baddest cloud around. That’s according to Adrian Cockroft, director of architecture of cloud systems for Netflix, who spoke on the topic at SVForum on Tuesday.

The AWS decision means that Netflix has not had to add data center capacity since 2008, according to Cockroft’s presentation on why Netflix built its platform as a service (PaaS) atop Amazon Web Services.

Netflix is a good example of a company that uses Amazon for what it does best — provide scale and power — but layers its own key technologies — in the form of a “thin” PaaS atop those Amazon services. However, Cockcroft’s presentation notes that Netflix would like its own PaaS to get thinner over time as Amazon moves up the stack.

The overriding reason for the Netflix move to a public cloud generally, and to AWS in particular, was that the company could not build data centers fast enough to meet the often-spiky demand of its users and that AWS was the only game in town in terms of the scale that Netflix needed.

One of the tweets from the SVForum event also piqued our curiosity because it indicates the pricing concessions — 33 percent off of  list — that Netflix, as a ginormous AWS customer, is able to negotiate.

Here are some key takeaways from Cockroft’s talk:

Why Netflix built its own PaaS atop AWS: Alternatives like VMware’s CloudFoundry are developer friendly but lack scale. RightScale, which many customers use as a management window into AWS, creates its own “lock-in” scenario, he said. Netflix didn’t want another vendor between it an AWS.

What that Netflix PaaS does:  Supports all AWS zones and regions; supports hundreds of developers on one project, monitors millions of metrics.

Core PaaS services used:  AWS S3 storage and Simple Queue Service; Netflix-based EVCache memcached ephemeral cache; Cassandra distributed data store. Some services like GeoIP lookup and Keystore HSM still run in the Netflix data center.

2012 roadmap: Work toward IPv6 support; add more automation and orchestration; prepare groundwork for global Netflix launches.

Why public versus private cloud?  Netflix needed the size and scale of a massive infrastructure and private clouds are “harder to build and support than you might think.”

Why the Amazon public cloud? AWS has more mature and less varied APIs, offers far more features faster, and much bigger scale than the “seven dwarf” options, Cockroft said. There may be a viable AWS alternative but not for “two or three years.”

Cockcroft also directly addressed the concern of many Amazon partners — that the platform provider is coming up the stack to compete with its PaaS customers. His response: The whole notion of “coopetition” between tech partners is as old as IT itself. In other words: Get over it. Netflix will be tied to AWS infrastructure for quite some time.

But these highlights are just a taste. Cloud tire kickers should check out the presentation itself as well as a longer version given at the qCon Conference in London earlier this month here , here and here.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

Tagged with: • • • • • • • • •
gen 29

5 low-profile startups that could change the face of big data

Generale, Hardware Upgrade, iPhone, Mobile No Comments »

Big data is hot, but infrastructure-level platforms such as Hadoop, which focus on storage and processing, still need help to take them into the mainstream. They need a killer app or two that will let companies analyze, visualize and act on all that data without hiring a team of Stanford Ph.Ds, or that will let developers write big-data apps without having to reinvent the wheel.

Here are five startups (in alphabetical order) either in stealth mode or just out of it that could help take Hadoop and its ilk to the promised land.

1. BloomReach

The stealth-mode BloomReach is taking a very targeted, very hands-free approach to big data for its customers. It’s offering a SaaS-based product that job listings say is for “helping leading online businesses uncover the highest quality, most relevant content sought by their consumers, when and where they want it.” Founded by a team with roots at Google, Cisco, Facebook and Yahoo, among other companies, BloomReach has, according to one estimate, about 160 customers — all of them among the top 10,000 websites, and most of them in the retail space. Among its core technologies and methods are Hadoop, Lucene, Monte Carlo simulations and large-scale image processing.

2. Continuuity

Continuuity, the just-launched stealth-mode startup by former Yahoo VP and chief cloud architect Todd Papaioannou, wants to make it easier to build applications that can leverage both cloud computing and big data technologies. As Papaioannou told me recently, most developers shouldn’t have to go through what Yahoo, Facebook and others did in order to write large-scale, data-driven applications. He also said “the data fabric is the next middleware” and noted that the company name is a play on “continuum.” You figure out what it’s up to.

3. Odiago

Odiago is the brainchild of Hadoop and analytics experts Christophe Bisciglia and Aaron Kimball, and aims to improve the state of web analytics. Its first product, Wibidata, which is in private beta, lets websites better analyze their user data to build more-targeted features. It’s built atop Hadoop and HBase, but also plugs into companies’ existing data-management and BI tools. Current customers include Wikipedia, RichRelavance, FoneDoktor and Atlassian (with whom it shares office space).

4. Platfora

Platfora, which launched in September with $5.7 million in funding, wants to make big data analytics accessible to the masses. Founder and CEO Ben Werther, formerly of Greenplum and NoSQL startup DataStax, told me when Platfora launched that its intuitive, visually stunning interface will make Hadoop-based analytics so easy even a history major could use it. Platfora’s product isn’t available yet, but the company is currently hiring, with an emphasis on frontend and user-experience skills.

5. SkyTree

Skytree is probably the stealthiest of the group, but it’s also is one of the more ambitious — because it’s trying to bring high-performance machine learning to mainstream companies. Machine learning is an impressive technique in which the system itself gets smarter as it digests more data, but it usually doesn’t find its way out of research environments or cutting-edge analytics teams. Skytree is putting together an impressive team, including co-founder Alexander Gray, who also teaches machine learning at Georgia Tech and spent six years at NASA’s Jet Propulsion Laboratory. The company will officially launch later this quarter.

Feature image courtesy of Flickr user jurvetson.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

Tagged with: • • • • • • • • • • • • • • • •
gen 12

Metamarkets takes its big-data-in-the-cloud message to the masses

Generale, Hardware Upgrade, iPhone, Mobile No Comments »

Metamarkets, (see disclosure) a San Francisco-based startup providing a cloud-based big data application, is readying itself for significant business growth by making key personnel changes and spreading its wings into new industries. The big personnel change is that co-founder and CTO Mike Driscoll is making the move to CEO, replacing fellow co-founder David Soloff, who will remain on the board while also taking on an entrepreneur in residence role at IA Ventures.

The Metamarkets product is a cloud-based big data application previously tuned for helping online media companies analyze the streams of data they generate everyday as customers click their ways through the sites. The company uses a specialized version of Hadoop for parallel processing, but goes much further by adding a custom-built in-memory database for real-time queries and by providing visualization and predictive modeling capabilities.

At a base level, Driscoll told me Metamarkets is ideal for a certain class of queries that can’t be done using a relational database — either for time or technical reasons — but that shouldn’t require a Ph.D. to perform. For example, he said, Metamarkets could help someone determine the top content from the last six hours for a certain demographic, easily and quickly even across a massive data set.

Because the platform itself has such broad applicability, Driscoll said Metamarkets is going to start extending it to also provide analytics for other industries, such as gaming and social media. Essentially, he said, it can handle pretty much anything that’s online and transaction-heavy if the proper algorithms and analysis methods are in place. Driscoll, who previously founded Dataspora, is joined in his mission to grow Metamarkets’ business by former Oracle executive Charlene Son Rigby and former MarkLogic executive Ken Chestnut, who will lead the company’s sales and marketing efforts.

One of the big criticisms about cloud-based big data platforms is that so much data is already stored on-premise, but with so much data now being generated on the web, that concern is becoming less important. Indeed, Metamarkets isn’t alone in targeting web companies’ analytics needs, and it might get even more competition if rumors of an Amazon Web Services analytics service  come to fruition. The company has its Elastic MapReduce processing service already, as well as its own analytic tools and a strategic investment in database startup ParAccel.

Disclosure: Metamarkets is a portfolio company of True Ventures, which is also an investor in GigaOM. Om Malik is also a venture partner at True.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

Tagged with: • • • • • • • •
gen 01

10 cloud startups to watch in 2012

Generale, Hardware Upgrade, iPhone, Mobile No Comments »

The past few years have been nothing if not a boon for entrepreneurs looking to cash in on venture capitalists’ lust for all things cloud.  All the activity has been great, and we’ve seen some exciting new companies emerge and prosper — companies such as Heroku, RightScale and New Relic — but it also means there’s precious little room on the playing field for newcomers. Startups that want to get noticed, get funded, and ultimately have a winning exit must either find their own unique niche or stake out ground on a different field altogether.

Here are 10 cloud computing startups that launched in 2011 and that have a chance to make it big in 2012.

1. AppFog 

AppFog is one of a handful of Platform-as-a-Service startups to launch in 2011, but AppFog is unique because it leverages the open-source Cloud Foundry code as its core. The switch to a Cloud Foundry foundation over the summer resulted in a name change from PHP Fog, as the company was immediately able to support numerous new programming languages. Going forward, AppFog can ride Cloud Foundry’s development wave, while focusing its own efforts on building the best user experience.

2. Bromium

Little is known about Bromium other than that is plans to use virtualization technology as a tool for securing the myriad endpoints (e.g., desktops, mobile phones and tablets) that connect to enterprise networks. While securing cloud servers, as other startups such as CloudPassage attempt to do, is important, the advent of consumerization means endpoints need security. Among Bromium’s founders is Simon Crosby, who co-founded XenSource and served as virtualization CTO at Citrix Systems.

3. Cloudability

Cloudability provides a simple service with a lot of value: it monitors customers’ spending on cloud computing resources. It might uncover something as commonplace as cloud-server sprawl because so many employees are spinning up instances, or it might find something nefarious such as hackers using a company’s instances serve boatloads of network traffic. As use of cloud services proliferates, companies will need an easy tool to help them keep track of what they’re spending and where.

4. CloudSigma 

The Infrastructure-as-a-Service space is a tough racket to enter because it means competing with the likes of Amazon Web Services and Rackspace, but CloudSigma has a plan. The company is all about giving customers high performance and lots of control. CloudSigma sits in the impressive SuperNAP data center and offers 10 GbE interconnects as well as solid-state drives, and developers can buy and manage resources with the granular control normally found in co-location.

5. Kaggle
Kaggle, a crowdsourcing platform for solving big data challenges, is about the hottest thing going in big data right now. The idea behind the service is simple: although not everyone has data scientists in-house, there are plenty of them floating around the world perfectly happy to put their skills to work on a problem for cash prizes and a little bit of credit. It takes a lot of computing power to host hundreds of teams on any given competition, as well as the data sets, which is why Kaggle utilizes Amazon Web Services.
6. Nebula 

Nebula isn’t the only company pushing a commercial version of the open-source OpenStack cloud computing software — it isn’t even the only one founded by a former NASA employee — but it does have a unique approach and an impeccable pedigree. Nebula ties OpenStack to an optimized hardware platform designed to make building public clouds a plug-and-play experience. Among its founders are former NASA CTO Chris Kemp, and investors include Andy Bechtolsheim, David Cheriton and Ram Shriram.

7. Parse

Parse is trying to become a PaaS specialist for mobile apps, a laudable ambition given how many people now rely on their mobile devices just about everything. It will be difficult to distinguish itself from competitors such as Stackmob, as well as from web-app PaaS offerings such as Heroku and AppFog, but Parse seems to have the right ideas in mind. It has a backend focused on the needs of mobile apps, and a frontend designed for mobile developers that might not have extensive programming chops.

8. ScaleXtreme

What ScaleXtreme lacks in sexiness it makes up for in functionality. Everyone needs server-management software, but not everyone needs the big, expensive software offered from traditional software vendors, or even wants to manage software at all. ScaleXtreme gives users a cloud-based service to manage both physical and cloud-based servers, and, it says, has also garnered a lot of interest from cloud providers thinking it might be a good value-added service to their users who want more control.

9. SolidFire

SolidFire wants nothing less than to revolutionize cloud computing by making it palatable to large enterprises wanting to run mission-critical applications. The company targets cloud providers with SSD-based storage systems that make it possible to store virtual machine images in the cloud and still deliver high performance. Cloud providers utilizing SolidFire gear could find themselves hosting far more relational databases and other applications that presently remain in house.

10. Zillabyte

Zillabyte, still operating in private beta mode, wants to provide users with both data sets and the algorithms needed to process them. Data sets aren’t uncommon on the web, but they usually don’t come with algorithms and a processing backend. The service will initially focus on web data and text-based algorithms, but there’s plenty of room for growth into new types of data and algorithms as the service matures. Zillabyte was co-founded by two former Google software engineers and a former Intel engineer.

Image courtesy of Flickr user JamesWoolley5.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

Tagged with: • • • • • • • • • • • • • • •
Previous Entries
 

Pages Menu 

 

Tags 

 

Archivi 

 

Categories 

 

Meta

preload preload preload