Welcome!

Log Management Authors: Dana Gardner, Pat Romanski, Elizabeth White, David H Deans, Carmen Gonzalez

Article

Inside story on developing the ultimate SDN-enabled hybrid cloud object storage environment

A new approach to open-source, hybrid cloud object storage

The next BriefingsDirect inside story interview explores how a software-defined data center (SDDC)-focused systems integrator developed an ultimate open-source object storage environment.

 

We’re now going to learn how Key Information Systems crafted a storage capability that may have broad extensibility into such realms as hybrid cloud and multi-cloud support.

 

Listen to the podcast. Find it on iTunes. Get the mobile app. Read a full transcript or download a copy.

 

Here to help us better understand a new approach to open-source object storage is Clayton Weise, Director of Cloud Services at Key Information Systems in Agoura Hills, California. The discussion is moderated by Dana Gardner, Principal Analyst at Interarbor Solutions.

 

Here are some excerpts:

 

Gardner: What prompted you to improve on the way that object storage is being offered as a service? How might this become a new business opportunity for you?

 

Weise: About a year ago, at Hewlett Packard Enterprise (HPE) Discover, I was wandering the event floor. We had just gotten out of a meeting with SwitchNAP, which is a major data center in Las Vegas. We had been talking to them about some preferred concepts and deployments for storage for their clients.

That discussion evolved into realizing that there are number of clients inside of Switch and their ecosystem that could make use of storage that was more locally based, that needed to be closer at hand. There were cost savings that could be gained if you have a connection within the same data center, or within the same fiber network.

 

Pulling data in and out of a cloud

Under this model, there would be significantly less expensive ways of pulling data in and out of a cloud, since you wouldn’t have transfer fees as you normally would. There would also be an advantage to privacy, and to cutting latency, and other beneficial things because of a private network all run by Switch and through their fiber network. So we looked at this and thought this might be interesting.

 

In discussions with the number of groups within HPE while wandering the floor at Discover, we found that there were some pretty interesting ways that we could play games with the network to allow clients to not have to uproot the way they do things, or force them to do things, for lack of a better term, “Our way.”

 

If you go to Amazon Web Services or you go to Microsoft Azure, you do it the Microsoft way, or you do it the Amazon way. You don’t really have a choice, since you have to follow their guidelines.

They generally use object storage as an inexpensive way to store archival or less-frequently accessed data. Cloud storage became an alternative to tape and long-term storage.

 

Where we saw value is, there are times in the mid-market space for clients -- ranging from a couple of hundred million dollars up to maybe a couple of billion dollars in annual revenue -- where they generally use object storage as kind of an inexpensive way to store archival, or less-frequently accessed, data. So [the cloud storage] became an alternative to tape and long-term storage.

 

We've had this massive explosion of unstructured data, files, and all sorts of things. We have a number of clients in medical and finance, and they have just seen this huge spike in data.

 

The challenge is: To deploy your own object storage is a fairly complex operation, and it requires a minimum number of petabytes to get started. In that mid-market, they are not typically measuring their storage in that petabytes level.

 

These customers are more typically in the tens to hundreds of terabytes range, and so they need an inexpensive way to offload that data and put it somewhere where it makes sense. In the medical industry particularly, there's a lot of concern about putting any kind of patient data up in a public cloud environment -- even with encryption.

 

We thought that if we are in the same data center, and it is a completely private operation that exists within these facilities, that will fulfill the total need -- and we can encrypt the data.

 

But we needed a way to support such private-cloud object storage that would be multitenant. Also, we just have had better luck working with open standards. The challenge with dealing with proprietary systems is you end up locked into a standard, and if you pick wrong, you find yourself having to reinvent everything later on.

 

I come from a networking background; I was an Internet plumber for many years. We saw the transition then on our side when routing protocols first got introduced. There were proprietary routing protocols, and there were open standards, and that’s what we still use today.

 

Transition to

HPE Data Center Networking

 

So we took a similar approach in object storage as a private-cloud service. We went down the open source path in terms of how we handled the provisioning. We needed something that integrated well with that. We needed a system that had the multitenancy, that understood the tenancy, and that is provided by OpenStack. We found a solution from HPE called Distributed Cloud Networking (DCN) that allows us to carve up the network in all sorts of interesting ways, and that way we don't have to dictate to the client how to run it.

 

Many clients are still running traditional networks. The adoption of Virtual Extensible LAN (VXLAN) and other types of SDDC within the network is still pretty low, especially in the mid-market space. So to go to a client and dictate that they have to change how they run the network it is not going to work.

 

And we wanted it to be as simple as possible. We wanted to treat this as much as we could as a flat network. By using a combination of DCN, Altoline switches from HPE, and some of other software, we were able to give clients a complete network carrying regular Virtual Local Area Networks (VLANs) across it. We then could tie this together in a hybrid fashion, whereby the customers can actually treat our cloud environment as a natural extension of their existing networks, of their existing data centers.

 

Gardner: You are calling this hybrid storage as a service. It’s focused on object storage at this point, and you can take this into different data center environments. What are some of the sweet spots in the market?

The object service becomes a very inexpensive way to store large amounts of data, and unlike tape -- with object as a service, everything is accessible easily.

 

Weise: The areas where we are seeing the most interest have been backup and archive. It’s an alternative to tape. The object service becomes a very inexpensive way to store large amounts of data, and unlike tape -- where it's inconvenient to access the data -- with object as a service everything is accessible very, very easily.

 

For customers that cannot directly integrate into that object service as supported by their backup software, we can make use of object gateways to provide a method that's more like traditional access. It looks like a file, or file share, and you edit the file share to be written to the object storage, and so it acts as a go-between. For backup and archive, it makes a really, really great solution.

 

The other two areas where we seen the most interest have been in the medical space, specifically for large medical image files and archival. We’re working now specifically to build that type of solution, with HIPAA compliance. We have gone through the audits and compliance verification.

 

The second use-case has been in the media and entertainment industry. In fact, they are the very first to consume this new system and put in hundreds of terabytes worth of storage -- they are an entertainment industry client in Burbank, California. A lot of these guys are just shuffling along on external drives.

 

For them it’s often external arrays, and it's a lot more Mac OS users. They needed something that was better, and so hybrid object storage as a service has created a great opportunity for them and allows them to collaborate.

 

They have a location in Burbank, and then they brought up another office in the UK. There is yet another office for them coming up in Europe. The object storage approach allows a kind of central repository, an inexpensive place to place the data -- but it also allows them to be more collaborative as well.

 

Gardner: We have had a weak link in cloud computing storage, which has been the network -- and you solved some of those issues. You found a prime use-case with backup and archival, but it seems to me that given the storage capabilities that we've seen that this has extensibility. So where it might go next in terms of a storage-as-a service that hybrid cloud providers would use? Where can this go?

 

Carving up the network

Weise: It’s an interesting question because one of the challenges we have all faced in the world of cloud is we have virtualized servers and virtualized storage, meaning there is disaggregation; there is a separation between the workload that’s running and the actual hardware it’s running on.

 

In many cases, and for almost all clients in the mid-market, that level of virtualization has not occurred at the network level. We are still nailed to things. We are all tied down to the cable, to the switch port, and to the human that can figure those things out. It’s not as flexible or as extensible as some of the other solutions that are out there.

 

In our case, when we build this out, the real magic is with the network. That improved connection might be a cost savings for a client -- especially from a bandwidth standpoint. But as you get a private cross-connect into that environment to make use of, in this case, storage as a service, we can now carve that up in a number of different ways and allow the client to use it for other things.

 

For example, if they want to have burst capability within the environments, they can have it -- and it’s on the same network as their existing system. So that’s where it gets really interesting: Instead of having to have complex virtual guest package configurations, and tiny networks, and dealing with some the routing of other pieces, you can literally treat our cloud environment as if it's a network cable thrown over the wall -- and it becomes just an extension of the existing network.

 

We can secure that traffic and ensure that there is high-performance, low-latency and complete separation of tenancy. If you have Coke and Pepsi as clients, they will never see each other.

That opens up some additional possibilities. Some things to work on eventually would be block storage, file storage, right there existing on the same network. We can secure that traffic and ensure that there is high-performance, low-latency and complete separation of tenancy. So if you have Coke and Pepsi as clients, they will never see each other.

 

Gardner: Very cool. You can take this object storage benefit -- and by the way, the cost of that can be significantly lower because you don’t have egress charges and some of the other unfriendly aspects of economics of public cloud providers. But you also have an avenue into a true hybrid cloud environment, where you can move data but also burst workloads and manage that accordingly. Now, what about making this work toward a multi-cloud capability?

 

Transition to

HPE Data Center Networking

 

Weise: Right. So this is where HPE’s DCN software-defined networking (SDN) really starts to shine and separates itself from the pack. We can tie environments together regardless of where they are. If there is a virtual endpoint or physical appliance; if it's at a remote location that can be deployed, which can act as a gateway -- that links everything together.

 

We can take a client network that's going from their environment into our environment, we can deploy a small virtual machine inside of a public cloud, and it will tie the networks together and allow them to treat it all as the same. The same policy enforcement engine and things that they use to segregate traffic in microsegmentation and service chaining can be done just as easily in the public cloud environment.

 

One of the reasons we went to Switch was because they have multiple locations. So in the case of our object storage, we deployed the objects across all three of their data center sites. So a single repository that’s written the data is distributed among three different regions. This protects against a possible regional outage that could mean data is inaccessible, and this is the kind of recent thing that we in the US have seen, where clients were down anywhere from 6 to 16 hours.

One big network, wherever you are

This eliminates that. But the nice thing is because of the network technology that theywere using from HPE, it allowed us to treat that all as one big network -- and we can carve that up and virtualize it. So clients inside of the data center -- maybe they need resources for disaster recovery or for additional backups or those things -- it's all part of that. We can tie-in from a network standpoint and regardless of where you want to exist -- if you are in Vegas, you may want to recover in Reno, or you may want to recover in Grand Rapids. We can make that network look exactly the same in your location.


You want to recover in AWS? You want to recover in Azure? We can tie it in that way, too. So it opens up these great possibilities that allows this true hybrid cloud -- and not as a completely separate entity.

 

Gardner: Very cool. Now there’s nothing wrong, of course, with Switch, but there are other fiber and data center folks out there. Some names that begin with “E” come to mind that you might want to drop in this and that should even increase the opportunity for distribution.

 

Weise: That’s right. So this initial deployment is focused on Switch, but we do a grand scheme to work this into other data centers. There are a handful of major data center operators out there, including the one that starts with an “E” along with another that starts with a “D.” We do have plans to expand this, or use this as a success use-case.

 

As this continues to grow, and we get some additional momentum and some good feedback, and really refine the offering to make sure we know exactly what everything needs to be, then we can work with those other data center providers.

 

Whenever clients deploy their workloads in those public clouds, that means there is equipment that has not been collocated inside one of your facilities.

From the data center operators’ perspective, if you're one of those facilities, you are at war with AWS or with Azure. Because whenever clients deploy their workloads in those public clouds, that means there is equipment that has not been collocated inside one of your facilities.

 

So they have a vested interest in doing this, and there is a benefit to the clients inside of those facilities too because they get to live inside of the ecosystem that exists within those data centers, and the private networks that they carry in there deliver the same benefits to all in that ecosystem.

 

We do plan to use this hybrid cloud object storage as a service capability as a model to deploy in several other data center environments. There is not only a private cloud, but also a multitenant private cloud that could be operative for clients that have a large enough need. You can talk about this in a multi-petabyte scale, or you talk about thousands of virtual machines. Then it's a question of should you do a private cloud deployment just for you? The same technology, fulfilling the same requirements, and the same solutions could still be used.

 

Partners in time

Gardner: It sounds like it makes sense, on the back of a napkin basis, for you and HPE to get together and brand something along these lines and go to market together with it.

 

Weise: It certainly does. We've had some great discussions with them. Actually there is a group that was popular in Europe that is now starting to take its growth here in US called Cloud28+.

 

We had some great discussions with them. We are going to be joining that, and it’s a great thing as well.

 

The goal is building out this sort of partner network, and working with HPE to do that has been extremely supportive. In addition to these crazy ideas, I also have a really crazy timeline for deployment. When we initially met with HPE and talked about what we wanted to do, they estimated that I should reserve about 6 to 8 weeks for planning and then another 1.5 months for deployment.

 

Transition to

HPE Data Center Networking

 

I said, “Great we have 3 weeks to do the whole thing,” and everyone thought we were crazy. But we actually had it completed in a little over 2.5 weeks. So we have a huge amount of thanks to HPE, and to their technical services group who were able to assist us in getting this going extremely quickly.

 

Listen to the podcast. Find it on iTunes. Get the mobile app. Read a full transcript or download a copy. Sponsor: Hewlett Packard Enterprise.

 

 

You may also be interested in:

 

More Stories By Dana Gardner

At Interarbor Solutions, we create the analysis and in-depth podcasts on enterprise software and cloud trends that help fuel the social media revolution. As a veteran IT analyst, Dana Gardner moderates discussions and interviews get to the meat of the hottest technology topics. We define and forecast the business productivity effects of enterprise infrastructure, SOA and cloud advances. Our social media vehicles become conversational platforms, powerfully distributed via the BriefingsDirect Network of online media partners like ZDNet and IT-Director.com. As founder and principal analyst at Interarbor Solutions, Dana Gardner created BriefingsDirect to give online readers and listeners in-depth and direct access to the brightest thought leaders on IT. Our twice-monthly BriefingsDirect Analyst Insights Edition podcasts examine the latest IT news with a panel of analysts and guests. Our sponsored discussions provide a unique, deep-dive focus on specific industry problems and the latest solutions. This podcast equivalent of an analyst briefing session -- made available as a podcast/transcript/blog to any interested viewer and search engine seeker -- breaks the mold on closed knowledge. These informational podcasts jump-start conversational evangelism, drive traffic to lead generation campaigns, and produce strong SEO returns. Interarbor Solutions provides fresh and creative thinking on IT, SOA, cloud and social media strategies based on the power of thoughtful content, made freely and easily available to proactive seekers of insights and information. As a result, marketers and branding professionals can communicate inexpensively with self-qualifiying readers/listeners in discreet market segments. BriefingsDirect podcasts hosted by Dana Gardner: Full turnkey planning, moderatiing, producing, hosting, and distribution via blogs and IT media partners of essential IT knowledge and understanding.

@ThingsExpo Stories
Data is the fuel that drives the machine learning algorithmic engines and ultimately provides the business value. In his session at Cloud Expo, Ed Featherston, a director and senior enterprise architect at Collaborative Consulting, discussed the key considerations around quality, volume, timeliness, and pedigree that must be dealt with in order to properly fuel that engine.
In his keynote at 19th Cloud Expo, Sheng Liang, co-founder and CEO of Rancher Labs, discussed the technological advances and new business opportunities created by the rapid adoption of containers. With the success of Amazon Web Services (AWS) and various open source technologies used to build private clouds, cloud computing has become an essential component of IT strategy. However, users continue to face challenges in implementing clouds, as older technologies evolve and newer ones like Docker c...
"Akvelon is a software development company and we also provide consultancy services to folks who are looking to scale or accelerate their engineering roadmaps," explained Jeremiah Mothersell, Marketing Manager at Akvelon, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
Two weeks ago (November 3-5), I attended the Cloud Expo Silicon Valley as a speaker, where I presented on the security and privacy due diligence requirements for cloud solutions. Cloud security is a topical issue for every CIO, CISO, and technology buyer. Decision-makers are always looking for insights on how to mitigate the security risks of implementing and using cloud solutions. Based on the presentation topics covered at the conference, as well as the general discussions heard between sessio...
To Really Work for Enterprises, MultiCloud Adoption Requires Far Better and Inclusive Cloud Monitoring and Cost Management … But How? Overwhelmingly, even as enterprises have adopted cloud computing and are expanding to multi-cloud computing, IT leaders remain concerned about how to monitor, manage and control costs across hybrid and multi-cloud deployments. It’s clear that traditional IT monitoring and management approaches, designed after all for on-premises data centers, are falling short in ...
Organizations planning enterprise data center consolidation and modernization projects are faced with a challenging, costly reality. Requirements to deploy modern, cloud-native applications simultaneously with traditional client/server applications are almost impossible to achieve with hardware-centric enterprise infrastructure. Compute and network infrastructure are fast moving down a software-defined path, but storage has been a laggard. Until now.
DXWorldEXPO LLC announced today that the upcoming DXWorldEXPO | CloudEXPO New York event will feature 10 companies from Poland to participate at the "Poland Digital Transformation Pavilion" on November 12-13, 2018.
Digital Transformation is much more than a buzzword. The radical shift to digital mechanisms for almost every process is evident across all industries and verticals. This is often especially true in financial services, where the legacy environment is many times unable to keep up with the rapidly shifting demands of the consumer. The constant pressure to provide complete, omnichannel delivery of customer-facing solutions to meet both regulatory and customer demands is putting enormous pressure on...
The best way to leverage your CloudEXPO | DXWorldEXPO presence as a sponsor and exhibitor is to plan your news announcements around our events. The press covering CloudEXPO | DXWorldEXPO will have access to these releases and will amplify your news announcements. More than two dozen Cloud companies either set deals at our shows or have announced their mergers and acquisitions at CloudEXPO. Product announcements during our show provide your company with the most reach through our targeted audienc...
In an era of historic innovation fueled by unprecedented access to data and technology, the low cost and risk of entering new markets has leveled the playing field for business. Today, any ambitious innovator can easily introduce a new application or product that can reinvent business models and transform the client experience. In their Day 2 Keynote at 19th Cloud Expo, Mercer Rowe, IBM Vice President of Strategic Alliances, and Raejeanne Skillern, Intel Vice President of Data Center Group and ...
More and more brands have jumped on the IoT bandwagon. We have an excess of wearables – activity trackers, smartwatches, smart glasses and sneakers, and more that track seemingly endless datapoints. However, most consumers have no idea what “IoT” means. Creating more wearables that track data shouldn't be the aim of brands; delivering meaningful, tangible relevance to their users should be. We're in a period in which the IoT pendulum is still swinging. Initially, it swung toward "smart for smart...
@DevOpsSummit at Cloud Expo, taking place November 12-13 in New York City, NY, is co-located with 22nd international CloudEXPO | first international DXWorldEXPO and will feature technical sessions from a rock star conference faculty and the leading industry players in the world.
DXWorldEXPO LLC announced today that All in Mobile, a mobile app development company from Poland, will exhibit at the 22nd International CloudEXPO | DXWorldEXPO. All In Mobile is a mobile app development company from Poland. Since 2014, they maintain passion for developing mobile applications for enterprises and startups worldwide.
In his keynote at 19th Cloud Expo, Sheng Liang, co-founder and CEO of Rancher Labs, discussed the technological advances and new business opportunities created by the rapid adoption of containers. With the success of Amazon Web Services (AWS) and various open source technologies used to build private clouds, cloud computing has become an essential component of IT strategy. However, users continue to face challenges in implementing clouds, as older technologies evolve and newer ones like Docker c...
DXWorldEXPO LLC announced today that ICC-USA, a computer systems integrator and server manufacturing company focused on developing products and product appliances, will exhibit at the 22nd International CloudEXPO | DXWorldEXPO. DXWordEXPO New York 2018, colocated with CloudEXPO New York 2018 will be held November 11-13, 2018, in New York City. ICC is a computer systems integrator and server manufacturing company focused on developing products and product appliances to meet a wide range of ...
Coca-Cola’s Google powered digital signage system lays the groundwork for a more valuable connection between Coke and its customers. Digital signs pair software with high-resolution displays so that a message can be changed instantly based on what the operator wants to communicate or sell. In their Day 3 Keynote at 21st Cloud Expo, Greg Chambers, Global Group Director, Digital Innovation, Coca-Cola, and Vidya Nagarajan, a Senior Product Manager at Google, discussed how from store operations and ...
Headquartered in Plainsboro, NJ, Synametrics Technologies has provided IT professionals and computer systems developers since 1997. Based on the success of their initial product offerings (WinSQL and DeltaCopy), the company continues to create and hone innovative products that help its customers get more from their computer applications, databases and infrastructure. To date, over one million users around the world have chosen Synametrics solutions to help power their accelerated business or per...
We are seeing a major migration of enterprises applications to the cloud. As cloud and business use of real time applications accelerate, legacy networks are no longer able to architecturally support cloud adoption and deliver the performance and security required by highly distributed enterprises. These outdated solutions have become more costly and complicated to implement, install, manage, and maintain.SD-WAN offers unlimited capabilities for accessing the benefits of the cloud and Internet. ...
Founded in 2000, Chetu Inc. is a global provider of customized software development solutions and IT staff augmentation services for software technology providers. By providing clients with unparalleled niche technology expertise and industry experience, Chetu has become the premiere long-term, back-end software development partner for start-ups, SMBs, and Fortune 500 companies. Chetu is headquartered in Plantation, Florida, with thirteen offices throughout the U.S. and abroad.
Dion Hinchcliffe is an internationally recognized digital expert, bestselling book author, frequent keynote speaker, analyst, futurist, and transformation expert based in Washington, DC. He is currently Chief Strategy Officer at the industry-leading digital strategy and online community solutions firm, 7Summits.