Media analytics and attribution are difficult to execute even when conditions are optimal. For most marketers, they’re typically a long way away from ‘optimal’: data volumes are enormous, data sits in multiple locations, and much of the data is redundant as well as disorganized.
Aqfer’s Marketing Data Platform-as-a-Service (MDPaaS) was built to work within – and correct – the situation as it exists for most marketers. MDPaaS was specifically designed to help those who serve marketers (and marketers themselves) more quickly and efficiently organize their marketing data in order to more effectively carry out analytics and attribution processes.
MDPaaS’ proprietary data format allows users to store data for longer periods of time compared to “similar” marketing data storage solutions.
MDPaaS can also run queries up to 98% faster than these similar solutions. This means that more of a marketer’s data is usable, connecting events over months becomes easier, and critical decisions get made faster and with more precision. With Aqfer, analytics become more insightful, and attributing credit becomes easier and more reliable.
Making Analytics With Aqfer Even Better
The marketing data management world is constantly evolving and adapting to new regulations and policies. Aqfer’s solutions and services are also constantly evolving in tandem with them as part of our commitment to helping our clients and their customers improve not just their analytics and attribution efforts, but also overall data efficiency and their bottom lines.
As part of Aqfer’s commitment to continuous innovation and improvement, we constantly engage our clients in ways that we can further enhance our solutions to address the challenges and inefficiencies they face in their day-to-day operations. So when one of our clients, a digital media operations platform, approached us about helping them to find a way to further reduce their query-related costs when using Google Campaign Manager 360, we jumped at the opportunity to make it happen.
The net result of this inquiry was a brand new commercial integration that drastically reduces GCM 360-related query costs for Aqfer MDPaaS users that utilize the platform as part of their analytics efforts.
Read on to learn the exact challenge that Aqfer helped the client to overcome, how and why Aqfer developed the commercial integration to solve that challenge, and why the commercial integration is so beneficial to current and future Aqfer clients.
Important Terms to Know Before You Dive In
To better understand how Aqfer has improved the data analytics efficiency for clients using Google Campaign Manager 360, it’s important to first define a few key technologies and products to give you clear insight into what the process involves. Sources for these definitions are noted.
Database query
A database query is either an action query or a select query. A select query is one that retrieves data from a database. An action query asks for additional operations on data, such as insertion, updating, deleting, or other forms of data manipulation.
This doesn’t mean that users just type in random requests. For a database to understand demands, it must receive a query based on the predefined code. That code is a query language, the most popular being Microsoft Structured Query Language (SQL).
Primarily, queries are used to find specific data by filtering explicit criteria. Queries also help automate data management tasks, summarize data, and engage in calculations.
In a relational database, which contains records or rows of information, the SQL SELECT statement query allows users to choose data and return it from a database to an application. The resulting query is stored in a result table, which is called a result set. Users can break down the SELECT statement into other categories. The SQL SELECT query can also group and aggregate data to analyze or summarize. Learn more…
Google Campaign Manager 360
Campaign Manager 360 (GCM 360) is an ad management and measurement system for advertisers and agencies that is part of the Google Marketing Platform.
GCM 360 helps users manage their digital campaigns across websites and mobile in a centralized location. This includes a robust set of features for ad serving, targeting, verification, and reporting. Learn more…
Google BigQuery
BigQuery is a fully-managed enterprise data warehouse that helps users manage and analyze their data with built-in descriptive and prescriptive analysis features like machine learning, geospatial analysis, and business intelligence.
BigQuery is native to and supports Google Cloud infrastructure. It can be considered Google’s equivalent to Amazon Athena.
BigQuery’s serverless architecture lets users use SQL queries to answer their organization’s biggest questions with zero infrastructure management. BigQuery’s scalable, distributed analysis engine lets users query terabytes in seconds and petabytes in minutes.
BigQuery maximizes flexibility by separating the compute engine that analyzes your data from your storage choices. You can store and analyze your data within BigQuery or use BigQuery to assess your data where it lives.
BigQuery interfaces include the Google Cloud console interface and the BigQuery command-line tool. Developers and data scientists can use client libraries with familiar programming including Python, Java, JavaScript, and Go, as well as BigQuery’s REST API and RPC API to transform and manage data. Learn more…
Amazon Athena (aka AWS Athena)
Amazon Athena is an interactive query service that makes it easy to analyze data directly in Amazon Simple Storage Service (Amazon S3) using standard SQL.
Athena is native to Amazon Web Services. It can be considered Amazon’s equivalent to Google BigQuery.
With a few actions in the AWS Management Console, users can point Athena at their data stored in Amazon S3 and begin using standard SQL to run ad-hoc queries and get results in seconds.
Athena also makes it easy to interactively run data analytics using Apache Spark without having to plan for, configure, or manage resources. When users run Apache Spark applications on Athena, they submit Spark code for processing and receive the results directly.
Athena SQL and Apache Spark on Amazon Athena are serverless, so there is no infrastructure to set up or manage, and users pay only for the queries they run. Athena scales automatically – running queries in parallel – so results are fast, even with large data sets and complex queries. Learn more…
Amazon Simple Storage Service (aka Amazon S3)
Amazon Simple Storage Service is an object storage service that offers industry-leading scalability, data availability, security, and performance. It is considered Amazon’s counterpart to Google Cloud Storage.
Customers of all sizes and industries can use Amazon S3 to store and protect any amount of data for a range of use cases, such as data lakes, websites, mobile applications, backup and restore, archive, enterprise applications, IoT devices, and big data analytics.
Amazon S3 provides management features so that users can optimize, organize, and configure access to their data to meet specific business, organizational, and compliance requirements. Learn more…
Aqfer Marketing Data Platform (aMDP) was originally built with Amazon S3 as its sole data storage service powering its data mart and data lake capabilities (clients using aMDP get their own secure Amazon S3 environment). In 2022, aMDP was enhanced to provide the option for users to utilize Google Cloud Storage for these same capabilities as well.
The Challenge That Needed to Be Solved
Many companies use Google Campaign Manager 360 for their web-based ad campaign management and tracking. By default, companies that utilize GCM 360 also use Google BigQuery for the data analysis, data segmentation, reporting (i.e., querying), and dashboards of these campaigns.
The issue with this is that Google BigQuery is significantly more expensive than most other data query services, even for simple queries. BigQuery’s pricing model is more challenging to predict and control because costs are based on both query usage and data storage, not just query usage.
How the MDPaaS-GCM 360 Commercial Integration Came To Be
In comparison to Google BigQuery, Amazon Athena – the query service native to Amazon S3 – provides lower and more predictable query costs because they factor in only query usage (and not data storage).
Why is this important? Amazon S3 is the underlying data storage technology found in MDPaaS that provides all Aqfer clients with their own unique, secure, virtual private cloud to perform data queries for analysis, reporting, and more. Because MDPaaS utilizes Amazon S3, this means that MDPaaS users could forgo using Google BigQuery and instead take advantage of Athena’s lower and more predictable query costs.
Simple solution, right? Well, not exactly. The data structures found in BigQuery (taken from the GCM 360 data) and Athena differ from one another, making it impossible to query GCM 360 data with Athena in a 1:1 manner. So the Product and CX Engineering teams at Aqfer got to work to develop a way to ingest and query GCM 360 data within MDPaaS (i.e., Amazon S3) with no hiccups or issues.
The end result of this work was an Aqfer-developed commercial integration that automatically and seamlessly ingests GCM 360 data and writes it to Amazon Athena for the same analysis, segmentation, reporting, dashboards, etc. available in Google BigQuery. This integration provides a data structure within Amazon S3 and Athena that was optimized to handle the data structure found in GCM and Google BigQuery. Views were also developed to mimic the GCM 360 data formats for minimal training.
This commercial GCM 360 integration for MDPaaS helps Aqfer clients avoid the high costs of using BigQuery to perform any type of data processing associated with their GCM 360 data. This ultimately provides Aqfer clients with significant savings in regard to their query costs. Costs using our GCM 360 integration for/with Amazon Athena can range from 1/5 to 1/10 of what they would be if they used BigQuery – while providing the same exact capabilities within Athena.
Ultimately, this integration provides Aqfer clients with an alternative to the higher and more unpredictable query costs associated with BigQuery and its proprietary storage system. It allows Aqfer clients to experience all the benefits of the GCM 360/BigQuery pairing while providing them with the lower and more predictable costs of Athena/S3.
The Major Benefits of the MDPaaS-GCM 360 Commercial Integration
In general, database querying and cloud computing storage are both quite expensive processes. As noted, BigQuery is natively more expensive than Athena because BigQuery users also incur a data storage cost when performing queries. Given the large amounts of data (billions of records in some cases), Aqfer’s clients are typically ingesting and processing, it’s easy to see why using MDPaaS (and its native S3 and Athena technology) over Google Cloud Infrastructure is the right move.
To the best of our knowledge, Aqfer is the only company that has a commercial integration (implementation) for GCM 360 data ingestion within Amazon Athena/S3. There are very few companies whose products can ingest and host the amount of data that MDPaaS is seamlessly able to do. MDPaaS can do this at a fraction of the cost that any other system can while at the same time servicing queries quickly and cost-effectively.
This commercial GCM 360 integration for MDPaaS helps Aqfer clients avoid the high costs of using BigQuery to perform any type of data processing associated with their GCM 360 data. This ultimately provides Aqfer clients with significant savings in regard to their query costs. Costs using our GCM 360 integration for/with Amazon Athena can range from 1/5 to 1/10 of what they would be if they used BigQuery – while providing the same exact capabilities within Athena.
Ultimately, this integration provides Aqfer clients with an alternative to the higher and more unpredictable query costs associated with BigQuery and its proprietary storage system. It allows Aqfer clients to experience all the benefits of the GCM 360/BigQuery pairing while providing them with the lower and more predictable costs of Athena/S3.
How Using Athena Over BigQuery Results in Substantial Cost Savings
Aqfer’s GCM 360 integration for use with Amazon Athena offers extremely similar query and analysis performance to that of Google BigQuery but at a fraction of the cost. The reasons for this come down to several key differences between BigQuery and Athena.
Athena’s pricing model is transparent and straightforward, allowing users to optimize costs by only paying for the data they query. BigQuery’s pricing model can be more challenging to predict and control because costs are based on both data storage and query usage.
Athena uses a serverless architecture, which automatically scales with the amount of data being processed. In comparison, BigQuery may require additional configuration and resources to achieve similar performance.
Athena uses Amazon S3 for storing data while BigQuery has its own proprietary storage system. S3 is a widely adopted, cost-effective, and versatile object storage service that offers various storage classes to suit different use cases.
Athena uses standard SQL, making it easier for users with existing SQL knowledge to adopt. On the other hand, BigQuery uses an SQL dialect called BigQuery SQL, which may require additional learning for users familiar with standard SQL.
Do More With Aqfer
Aqfer continues to revolutionize how MSPs support today’s data-driven marketers and advertisers. Our MDPaaS makes it faster, easier, and cheaper to overcome today’s most pressing marketing data collection and management challenges. As demonstrated by such Innovations as the MDPaaS-Google Ad Manager integration outlined here, Aqfer is committed to constantly improving our products to ensure our clients have the tools they need to help their customers overcome these challenges both now and in the future.
Interested in learning more about how Aqfer’s solutions have supported our clients? Visit our Practical Applications resource page for more detailed insight into how our clients have used the Aqfer MDPaaS to enhance identity resolution, universal tag management, analytics and attribution, and more.