Gains Scalability Required For Large Hadoop Development and Test Environments
- Finite IT resources for dev/test environments
- Provisioning and managing complex environments for software testing
- Self-service and ease of use for dev/test teams to get environments quickly
- Manual management of network IP addressing for on-premises and cloud resources
- Zettaset adopted Skytap Cloud to gain the scalability required to develop and test its software solutions across large Hadoop clusters, and to simplify accessibility to environments for developers, test engineers, regional sales teams,
- Scalability from finite, on-premises resources to public cloud for dev/test environments
- Time saved with rapid provisioning and scalability for testing on large Hadoop Clusters
- Simplified environment management across application development and testing teams with visibility and control for IT
- Use of Network Address Translation (NAT) to simplify management of complex network routes and IP addressing in a Hybrid Cloud architecture
Zettaset is an enterprise software company and the creator of Orchestrator™, an enterprise-ready Hadoop cluster management solution designed for organizations that need a more automated, efficient, and reliable way to structure and manage big data. Orchestrator significantly improves Hadoop cluster security, availability and performance, and is designed to manage any Apache Hadoop distribution.
Zettaset selected Skytap Cloud to gain the massive scalability it requires to test its solutions against large Hadoop clusters, and to take advantage of Skytap Cloud’s rapid provisioning and management of complex environments. The move to Skytap Cloud has saved Zettaset from expanding its on-premises VMware infrastructure, and the ease of use has enabled it to keep its development and testing teams focused on product development, instead of managing infrastructure.
“I didn’t have to ask engineering to change their workflow. Everything that we were doing in our own VMware environment we can do in Skytap Cloud, only faster and better. No changes whatsoever were required, everything worked perfectly.”Timothy Demarest
Senior Director of Technical Operations
With big data becoming an increasingly important resource, many organizations have supplemented or replaced their traditional data warehouses to embrace the massive scalability of the non-relational Hadoop system. Zettaset was founded to resolve the problems that organizations soon face after deploying Hadoop—the lack of management and administrative tools required by the enterprise.
Zettaset is a big data software company that provides a services management layer on top of Hadoop distributions. In their own words, “We make Hadoop Enterprise Ready.” An important element of making Hadoop enterprise-ready is Orchestrator’s ability to help ensure compliance requirements are addressed. Zettaset supplements leading Hadoop distributions with more robust security, encryption, and authentication services such as role-based access control and LDAP support.
It’s no surprise that big data requires big resources to support development and testing. “Initially developing and testing on small Hadoop environments that were no more than 3- to 4-node clusters was fine,” says Timothy Demarest, Senior Director of Technical Operations at Zettaset. “But as the business scaled, doing adequate testing on larger node clusters was nearly impossible.”
The need to support environment templates for consistency added to the difficulties. “We were really struggling with sharing complex environments, builds, and QA environments,” Demarest says. “The efficient sharing of standardized environments was extremely challenging. We needed a better solution to keep up with customer demand.”
As the popularity of Orchestrator grew, Zettaset’s rapid success soon created problems with scalability. The company’s twenty-five developers and five test engineers needed the ability to simulate larger Hadoop clusters. Initially, their development and test needs were met by deploying an on-premises VMware infrastructure. However, as the company continued to grow, they were faced with the need to either continue building out more capacity or find a cloud-based solution to support their growth and needs for greater scalability, and to support a geographically dispersed sales team with product demo infrastructure needs.
One of the challenges the company faced was creating, self provisioning, cloning, and sharing environments that Zettaset used mainly for development and testing. The company deployed Amazon Web Services resources and considered Rackspace, but found these solutions to be too complex and difficult to use. Zettaset wanted to keep its focus on core competencies—creating better software faster.
Zettaset chose Skytap Cloud as its solution and because of the early successes is exploring moving more workloads from its on-premises VMware infrastructure and AWS to Skytap Cloud. The decision came after the company ran some experiments with Amazon Web Services, Rackspace, and Skytap Cloud. Zettaset discovered that Skytap Cloud best met its needs and found that the repeatability of templatized complete environments, resource sharing, and speed of provisioning from other providers didn’t compare with what Skytap Cloud offers. They were looking for push-button simplicity for complex computing environments, and found what they wanted with Skytap Cloud.
Zettaset is especially pleased with the ease with which Skytap Cloud supports templates, and notes that no engineering was required to integrate their existing VMware assets and workflows with Skytap Cloud. Zettaset also takes advantage of the Skytap Cloud network address translation (NAT) feature, which allows Zettaset to easily create multiple parallel development and test environments without dealing with the networking issues that commonly occur in shared networks across hybrid cloud architectures.
Zettaset has enjoyed a number of benefits since moving dev/test workloads to Skytap Cloud, including instant, self-service access to environments for engineering and QA teams, sharing complex environments easily, scalability for massive testing and easier environment management.
200% Faster Testing Using Skytap API
Using the Skytap Cloud API to automate testing has proven to be a major benefit for Zettaset, allowing it to slash the manual steps that used to be involved and to increase their ability to perform multiple tests in parallel.
“The Skytap Cloud API has been a key feature for us because its eliminated about 90% of the manual steps to get templates uploaded into the environment, and has allowed us to move more and more automated testing from our data center into Skytap Cloud,” Demarest says. “This has allowed us to test multiple releases simultaneously.”
Slashing the need for manual steps and gaining the ability to test different builds concurrently has had a big impact on productivity. “Previously these steps were very manual and we could only test different releases serially,” Demarest says. “We’ve been able to test builds with results close to 200% improvement in time to completion.”
Scalability for Massive Testing
Skytap Cloud gives Zettaset the scalability it requires for testing Hadoop solutions. “We used to be limited to working with 3 to 4 node clusters,” Demarest says. “Using Skytap Cloud we can now test our product on 50-node clusters, and go even higher if ever needed. This scalability is awesome, and it is something we couldn’t have done effectively with our current VMware on-premises datacenter.”
Easier Environment Management
Prior to working with Skytap Cloud, environment management was an ongoing challenge for Zettaset, particularly as its sales force expanded to keep up with market demand for its products. “Our sales team had a standardized environment for demonstrations and evaluations,” Demarest says. “But as our sales team grew, maintaining these different environments on-premises was taxing. Skytap Cloud alleviates that all together.”
Development and test experienced similar problems. The groups began using OVA files and would pass environment templates around using USB drives. But the company faced an ongoing challenge of maintaining consistency over users and time. “With Skytap Cloud we have templates of multiple environment configurations that match different builds,” Demarest says. “Having everyone working from the same base templates for our workloads and test environments has been huge.”
Simplicity of Deployment and Use
Zettaset values the ease with which they’ve been able to move workloads to Skytap Cloud and seamlessly use the resources. “I didn’t have to ask engineering to change their workflow,” Demarest says. “Everything that we were doing in our own VMware environment we can do in Skytap Cloud, only faster and better. No changes whatsoever were required, everything worked perfectly.”
The ease of use is especially important to a company like Zettaset with a sharp focus on product development. “Our [technical operations] department is very small so we don’t have a lot of people to help automate the provisioning of environments,” Demarest says. “To off-load these concerns and leverage a solution like Skytap Cloud, allows us to concentrate on releasing software faster and selling a better product.”
Use of NAT to Simplify Network Addressing
Zettaset was relieved to be able to take advantage of Skytap Cloud NAT services. “The Skytap Cloud NAT feature eliminates the need for us to manually manage network routes and IP addressing,” Demarest says. “This allows Zettaset to easily create multiple parallel development and test environments without dealing with the networking issues that commonly occur in shared networks across hybrid cloud architectures.”