Welcome!

Log Management Authors: David H Deans, Carmen Gonzalez, Eric Robertson, Liz McMillan, Pat Romanski

Related Topics: Containers Expo Blog, Log Management, @CloudExpo, @BigDataExpo

Containers Expo Blog: Blog Post

Big Data Analysis in Ad Marketing By @Dana_Gardner | @CloudExpo [#BigData]

adMarketplace solves search intent challenge with HP Vertica big data warehouse

The Power of Big Data Analysis in Ad Marketing

The next BriefingsDirect bid data trailblazer interview examines how New York-based adMarketplace, a search syndication advertising network, has met its daunting data-warehouse requirements.

Learn here how adMarketplace captures and analyzes massive data to allow for efficient real-time bidding for traffic sources for online advertising. And we'll hear how the data-analysis infrastructure also delivers rapid cost-per-click insights to advertisers.

For the inside story, BriefingsDirect sat down with Michael Yudin, the Chief Technology Officer at adMarketplace at the recent  HP Discover 2014 Conference in Las Vegas. The discussion is moderated by me, Dana Gardner, Principal Analyst at Interarbor Solutions.

Here are some excerpts:

Gardner: Tell us first about what adMarketplace does. It sounds very interesting, but I'm not sure I fully understand it.

Yudin: Well, adMarketplace is the leading marketplace for search intent advertising, and let me explain what that means. Search advertising is the best form of advertising ever invented. For the first time, a consumer actually tells a computer what they're interested in. That’s why Google became so successful as a search engine.

Yudin

Some things are changing in the marketplace these days. Consumer search intent is fracturing. You probably wonder what this means. It’s very simple. What this means is Google is no longer the only place you go to search for stuff.

I'll give you an example. Last night, I was looking for a Brazilian steakhouse here in Las Vegas. I didn't go on google.com. I opened my iPhone and I fired up a yellow pages (YP) app and I entered "Brazilian steakhouse" in the search box.

There are a variety of apps in my phone like that for travel, sports, news, and various other things I'm interested in. Anytime I search there, I don’t go to google.com. Consumer search has really fractured and adMarketplace has solved the monetization problem for that.

Providing value

Gardner: So when people are searching in areas other than say Google or Yahoo, how does your organization intercept with that and how does that provide value to both the consumer that’s searching and advertisers that want to provide them information?
Yudin: It benefits both the consumer and the advertiser. In the search world, an ad is really nothing more than a search result in response to user’s query. That’s why it’s so great.

Our clients are the Internet's largest marketers and brands. They use adMarketplace to acquire additional customers in addition to the other marketing channels like Google, where they are pretty much already maxed out.

http://bit.ly/1En8DHKThere are only so many searches that happen in Google and they're declining. So advertisers are looking for new ways to capture consumer intent and to convert this into sales and measurable return on investment (ROI), and that's what we do for them.

Gardner: Of course, a really important thing here is to match properly, and that requires data and analysis -- and it requires speed. Tell us a little about the requirements. How do you do this technically?

Yudin: You just nailed it. This is a very, very big data problem and it has to be solved at scale and fast. And it’s also a 24x7 problem. We can never take our system down. We have a global business, and anytime you go and you search for something as a consumer, you expect to see the result right away.

Our network handles about half a billion search queries per day and this results in about two terabytes of data per hour constantly generated by our platform, across multiple data centers. We needed a very scalable and robust analytical data warehouse solution that could handle this. Two years ago, we evaluated a number of vendors and settled on HP Vertica, which was best able to satisfy our tough requirements.

Gardner: And are these requirements primarily about the scale and volume, or are we talking also about a need for rapid query, or all the above? Give us a bit more insight into the actual requirements for your network?

Yudin: That's a great question, and I think this is what makes Vertica unique. There are products out there that can store a lot of data, but you can't get this data out of these solutions quickly and at high concurrency. We require a system that can ingest large amounts of data constantly. I am talking about terabytes and terabytes of data. This data has to be queryable right away, with very low latency requirements.

Some of our queries for Advertiser 3D and analytical dashboard are preplanned queries obviously, but they are very big data queries and the service-level agreement (SLA) on these queries is two seconds. Very few products can do that. Some queries are obviously more complex, but we're still talking about seconds and not hours.

Concurrency requirement

On top of this, there's a concurrency requirement and that’s a very big weak spot of a lot of products. Vertica is actually able to provide sufficient concurrency, and it’s never enough.

I do know that there's an upcoming release of Vertica 7, where this is going to be improved even further, but it’s quite acceptable right now. And it has to be fault tolerant, which means that it should be able to sustain a hardware failure on any of its nodes -- and it can do that.

Gardner: Tell us a bit about where you've built Vertica in terms of data centers. Are they your own? Do you have managed service providers? How are you managing your infrastructure that supports Vertica and then therefore your data processes?

Yudin: We own our own infrastructure. So these are not managed services. We actually once used managed services, but we've outgrown them. And Vertica runs on dedicated hardware.

This was driven by business requirements. We didn’t just decide that we needed this

We also have several other Vertica clusters that run on virtualized hardware, and even though it’s dedicated infrastructure, it’s really dedicated at the cloud level now. So call it private cloud. It's a buzzword. It's a mix of dedicated and virtualized. It's elastic scaling.

Gardner: And the transition. You mentioned that two years ago, you were searching for a product. How were you able to bring this on board and what sort of growth have you had as a result -- in terms of data volume, but also in your business, in terms of customers and overall business metrics of growth?

Yudin: This was driven by business requirements. We didn’t just decide that we needed this. So we started to undertake a very, very ambitious project -- Advertiser 3D. If you go to our website, www.admarketplace.com, you can read more about it.

This is a very elegant, simple, and yet powerful, system to match and price traffic across a multitude of traffic sources. To deliver this product, we didn’t have a choice. We had to have a powerful analytical back-end data warehouse. That's when we started to evaluate products and chose Vertica.

Gardner: And have there been any other benefits of going to Vertica in terms of being able to increase the number of features, or have you been able to leverage the technology in new business opportunities in terms of what you can offer your customers, not just to have met the requirements, but perhaps whole new types of benefits?

Yudin: Definitely. Our customers don’t know and don’t even care that we use Vertica on the back end. That’s probably why we won an HP award, because we integrated it into our overall solution very elegantly and seamlessly, but it obviously does a lot of heavy lifting on the back end.

And the project was successful and transformed our business. Our growth rates have accelerated over 50 percent on our core revenue and performance. Data-savvy marketers, and our clients started to see significantly double-digit improvement in ROIs.

Gardner: As Chief Technology Officer there, you've gone through a fairly significant change in your infrastructure and adoption, as you've just described. Looking back, are there any lessons learned that you could offer to others who are also running into a wall with their data infrastructure or looking for alternatives? Any thoughts on how you would advise them to make the transition?

Yudin: Definitely. The number one advice I would give anybody is don’t believe anything until you do two things: Try it yourself and get references from people who actually use this and whom you trust. That's very important.

You may also be interested in:

More Stories By Dana Gardner

At Interarbor Solutions, we create the analysis and in-depth podcasts on enterprise software and cloud trends that help fuel the social media revolution. As a veteran IT analyst, Dana Gardner moderates discussions and interviews get to the meat of the hottest technology topics. We define and forecast the business productivity effects of enterprise infrastructure, SOA and cloud advances. Our social media vehicles become conversational platforms, powerfully distributed via the BriefingsDirect Network of online media partners like ZDNet and IT-Director.com. As founder and principal analyst at Interarbor Solutions, Dana Gardner created BriefingsDirect to give online readers and listeners in-depth and direct access to the brightest thought leaders on IT. Our twice-monthly BriefingsDirect Analyst Insights Edition podcasts examine the latest IT news with a panel of analysts and guests. Our sponsored discussions provide a unique, deep-dive focus on specific industry problems and the latest solutions. This podcast equivalent of an analyst briefing session -- made available as a podcast/transcript/blog to any interested viewer and search engine seeker -- breaks the mold on closed knowledge. These informational podcasts jump-start conversational evangelism, drive traffic to lead generation campaigns, and produce strong SEO returns. Interarbor Solutions provides fresh and creative thinking on IT, SOA, cloud and social media strategies based on the power of thoughtful content, made freely and easily available to proactive seekers of insights and information. As a result, marketers and branding professionals can communicate inexpensively with self-qualifiying readers/listeners in discreet market segments. BriefingsDirect podcasts hosted by Dana Gardner: Full turnkey planning, moderatiing, producing, hosting, and distribution via blogs and IT media partners of essential IT knowledge and understanding.

@ThingsExpo Stories
China Unicom exhibit at the 19th International Cloud Expo, which took place at the Santa Clara Convention Center in Santa Clara, CA, in November 2016. China United Network Communications Group Co. Ltd ("China Unicom") was officially established in 2009 on the basis of the merger of former China Netcom and former China Unicom. China Unicom mainly operates a full range of telecommunications services including mobile broadband (GSM, WCDMA, LTE FDD, TD-LTE), fixed-line broadband, ICT, data communica...
As businesses adopt functionalities in cloud computing, it’s imperative that IT operations consistently ensure cloud systems work correctly – all of the time, and to their best capabilities. In his session at @BigDataExpo, Bernd Harzog, CEO and founder of OpsDataStore, will present an industry answer to the common question, “Are you running IT operations as efficiently and as cost effectively as you need to?” He will expound on the industry issues he frequently came up against as an analyst, and...
SYS-CON Events announced today that CA Technologies has been named "Platinum Sponsor" of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, New York, and 21st International Cloud Expo, which will take place in November in Silicon Valley, California.
SYS-CON Events announced today that delaPlex will exhibit at SYS-CON's @CloudExpo, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. delaPlex pioneered Software Development as a Service (SDaaS), which provides scalable resources to build, test, and deploy software. It’s a fast and more reliable way to develop a new product or expand your in-house team.
With major technology companies and startups seriously embracing IoT strategies, now is the perfect time to attend @ThingsExpo 2016 in New York. Learn what is going on, contribute to the discussions, and ensure that your enterprise is as "IoT-Ready" as it can be! Internet of @ThingsExpo, taking place June 6-8, 2017, at the Javits Center in New York City, New York, is co-located with 20th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry p...
The explosion of new web/cloud/IoT-based applications and the data they generate are transforming our world right before our eyes. In this rush to adopt these new technologies, organizations are often ignoring fundamental questions concerning who owns the data and failing to ask for permission to conduct invasive surveillance of their customers. Organizations that are not transparent about how their systems gather data telemetry without offering shared data ownership risk product rejection, regu...
WebRTC sits at the intersection between VoIP and the Web. As such, it poses some interesting challenges for those developing services on top of it, but also for those who need to test and monitor these services. In his session at WebRTC Summit, Tsahi Levent-Levi, co-founder of testRTC, reviewed the various challenges posed by WebRTC when it comes to testing and monitoring and on ways to overcome them.
Every successful software product evolves from an idea to an enterprise system. Notably, the same way is passed by the product owner's company. In his session at 20th Cloud Expo, Oleg Lola, CEO of MobiDev, will provide a generalized overview of the evolution of a software product, the product owner, the needs that arise at various stages of this process, and the value brought by a software development partner to the product owner as a response to these needs.
The Internet of Things can drive efficiency for airlines and airports. In their session at @ThingsExpo, Shyam Varan Nath, Principal Architect with GE, and Sudip Majumder, senior director of development at Oracle, discussed the technical details of the connected airline baggage and related social media solutions. These IoT applications will enhance travelers' journey experience and drive efficiency for the airlines and the airports.
In his keynote at @ThingsExpo, Chris Matthieu, Director of IoT Engineering at Citrix and co-founder and CTO of Octoblu, focused on building an IoT platform and company. He provided a behind-the-scenes look at Octoblu’s platform, business, and pivots along the way (including the Citrix acquisition of Octoblu).
In his session at @ThingsExpo, Sudarshan Krishnamurthi, a Senior Manager, Business Strategy, at Cisco Systems, will discuss how IT and operational technology (OT) work together, as opposed to being in separate siloes as once was traditional. Attendees will learn how to fully leverage the power of IoT in their organization by bringing the two sides together and bridging the communication gap. He will also look at what good leadership must entail in order to accomplish this, and how IT managers ca...
In his keynote at @ThingsExpo, Chris Matthieu, Director of IoT Engineering at Citrix and co-founder and CTO of Octoblu, focused on building an IoT platform and company. He provided a behind-the-scenes look at Octoblu’s platform, business, and pivots along the way (including the Citrix acquisition of Octoblu).
SYS-CON Events announced today that CA Technologies has been named “Platinum Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY, and the 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. CA Technologies helps customers succeed in a future where every business – from apparel to energy – is being rewritten by software. From ...
SYS-CON Events announced today that Outlyer, a monitoring service for DevOps and operations teams, has been named “Bronze Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Outlyer is a monitoring service for DevOps and Operations teams running Cloud, SaaS, Microservices and IoT deployments. Designed for today's dynamic environments that need beyond cloud-scale monitoring, we make monitoring effortless so you...
DevOps is being widely accepted (if not fully adopted) as essential in enterprise IT. But as Enterprise DevOps gains maturity, expands scope, and increases velocity, the need for data-driven decisions across teams becomes more acute. DevOps teams in any modern business must wrangle the ‘digital exhaust’ from the delivery toolchain, "pervasive" and "cognitive" computing, APIs and services, mobile devices and applications, the Internet of Things, and now even blockchain. In this power panel at @...
In his session at @ThingsExpo, Steve Wilkes, CTO and founder of Striim, will delve into four enterprise-scale, business-critical case studies where streaming analytics serves as the key to enabling real-time data integration and right-time insights in hybrid cloud, IoT, and fog computing environments. As part of this discussion, he will also present a demo based on its partnership with Fujitsu, highlighting their technologies in a healthcare IoT use-case. The demo showcases the tracking of pati...
SYS-CON Events announced today that Cloud Academy will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Cloud Academy is the industry’s most innovative, vendor-neutral cloud technology training platform. Cloud Academy provides continuous learning solutions for individuals and enterprise teams for Amazon Web Services, Microsoft Azure, Google Cloud Platform, and the most popular cloud computing technologies. Ge...
With 10 simultaneous tracks, keynotes, general sessions and targeted breakout classes, Cloud Expo and @ThingsExpo are two of the most important technology events of the year. Since its launch over eight years ago, Cloud Expo and @ThingsExpo have presented a rock star faculty as well as showcased hundreds of sponsors and exhibitors! In this blog post, I provide 7 tips on how, as part of our world-class faculty, you can deliver one of the most popular sessions at our events. But before reading the...
In his keynote at 18th Cloud Expo, Andrew Keys, Co-Founder of ConsenSys Enterprise, provided an overview of the evolution of the Internet and the Database and the future of their combination – the Blockchain. Andrew Keys is Co-Founder of ConsenSys Enterprise. He comes to ConsenSys Enterprise with capital markets, technology and entrepreneurial experience. Previously, he worked for UBS investment bank in equities analysis. Later, he was responsible for the creation and distribution of life settle...
A strange thing is happening along the way to the Internet of Things, namely far too many devices to work with and manage. It has become clear that we'll need much higher efficiency user experiences that can allow us to more easily and scalably work with the thousands of devices that will soon be in each of our lives. Enter the conversational interface revolution, combining bots we can literally talk with, gesture to, and even direct with our thoughts, with embedded artificial intelligence, whic...