BigQuery Lakehouse Caching GA and SECURE_CONTEXT Preview

BigQuery's release notes for 2 November 2026 make cross-cloud caching for Lakehouse data generally available, add a Preview of a new SECURE_CONTEXT function, and take AI.PARSE_DOCUMENT, data agent monitoring and Data Engineering Agent support for AI and ML functions to GA. The one most teams should look at first is caching, because it touches both cost and query speed for data that lives outside Google Cloud. None of it switches on by itself.

Key takeaways

• Lakehouse caching for cross-cloud data access is GA, but it is disabled by default. You opt in per federated catalog or cross-cloud connection.
• The cache holds remote files in your Google Cloud region, which the notes say reduces cross-cloud egress costs and speeds up subsequent queries.
• AI.PARSE_DOCUMENT is GA and converts layout elements from documents into structured JSON through the Document AI layout parser.
• SECURE_CONTEXT is Preview. Use it to prototype tenant-aware logical views, not yet as the only guard on sensitive data.
• Agent monitoring in Google Cloud Observability and the Data Engineering Agent update are GA and matter mainly to teams already using those agents.
• Start with one change, record a baseline, and bring in outside help only where the design is hard to undo.

What did Google change in BigQuery on 2 November 2026?

Five items landed in the same set of notes. Three are generally available features that extend things teams may already be using. One is a Preview. One is a monitoring capability for data agents.

Here is the short version, with who should care and what to do first.

Change	Status	Who it affects	First action
Lakehouse caching for cross-cloud data access	GA, off by default	Teams querying data held in another cloud	Pick one repeatedly queried remote table and trial the cache
Monitoring of data agents in Google Cloud Observability	GA	Teams running data agents and their conversations	Open the monitoring view and note latency, adoption and cost
AI.PARSE_DOCUMENT	GA	Teams with PDFs, reports and other unstructured documents	Run it on a small sample of real documents
SECURE_CONTEXT	Preview	Teams building views for multiple departments or tenants	Prototype on a non-sensitive dataset
Data Engineering Agent support for BigQuery AI, ML and object functions	GA	Teams generating Dataform SQLX pipelines with the agent	Try one pipeline that uses an AI function

The notes are terse on all five. Where this post says what something is for, that is background on how BigQuery is generally used. Confirm limits, supported sources and pricing details in Google's documentation before you commit.

Should you turn on Lakehouse caching for cross-cloud data?

Probably for some tables, not for all of them. That is the honest answer, and the default tells you why. Google has left caching off, so you decide where it earns its place.

The mechanism, as the notes describe it, is simple. You opt in on a Lakehouse federated catalog or on a BigQuery cross-cloud connection. After that, remote files can be cached in your Google Cloud region. The stated benefits are lower cross-cloud egress costs and faster subsequent queries.

Notice the word subsequent. A cache helps when the same files are read more than once. Picture a marketing analytics team whose click and order exports sit in another cloud's object storage. A dashboard hits the last 30 days of that data every morning, and analysts rerun similar queries all day. That pattern is a good fit. Now picture a data scientist who scans a huge historical archive once for a one-off study. The cache gives that person little, because nothing is read twice.

So the selection test is about reads, not size. Look for remote data that is queried often, queried by many people, or queried by scheduled jobs.

A sensible trial looks like this:

1. List the remote tables and catalogs your team queries, and rank them by how often they are read.
2. Pick one with a repeated read pattern and a known owner.
3. Record a baseline first: typical query duration and the cross-cloud transfer line on your bill for that workload.
4. Enable caching on that catalog or connection only.
5. Rerun the same workload for a couple of weeks and compare.

What the notes do not give you is a freshness rule, a cache size or a price for the cached storage in your region. Don't assume them. If your remote files change often, ask how stale a cached read can be before you rely on it for reporting that finance or leadership sees. That question belongs in the documentation and, if it matters, a test with a file you change on purpose.

How do you measure whether caching is paying off?

Two numbers, taken before and after: query time and egress cost. Everything else is commentary.

For query time, pick five to ten queries that represent the workload and time them cold and then warm. The first run after enabling caching may not look better. The later ones should. If they don't, the cache isn't being used the way you expected, or the data isn't being read repeatedly.

For cost, find the cross-cloud transfer charges attached to the workload in your billing export and compare the same period length before and after. Direction is what you're looking for. A drop in transfer cost with no change in query volume is a clean win. A flat line means you chose the wrong table.

One caution. Don't judge it on a single week that includes a holiday or a campaign spike. Usage swings will drown out the effect. Compare like for like.

What does AI.PARSE_DOCUMENT add for teams with unstructured files?

It gives you a SQL-side way to turn documents into structure. The function uses the Document AI layout parser to extract elements such as titles, headings, paragraphs, lists, tables and images, and returns them as structured JSON. It is GA.

Who benefits? Think of teams sitting on piles of agency reports, vendor contracts, campaign briefs or compliance PDFs. Until now, getting the headings and tables out of those usually meant a separate pipeline with its own code and scheduling. Doing it from BigQuery keeps the parsed output next to the rest of your data, where you can join it, filter it and hand it to analysts.

The practical value is in the layout. A flat text dump loses which sentence was a heading and which cell belonged to which table. Layout-aware JSON keeps that, so you can pull only the tables, or only the sections under a given heading.

Try it small. Take 20 real documents, not clean samples, and include the ugly ones: scans, multi-column layouts, tables that span pages. Then read the JSON against the originals. Count how many tables came out intact. That count tells you whether you can automate or whether you still need a person checking.

And don't skip the governance question. Documents often contain personal or commercial data. Before you parse a folder in bulk, decide who may see the output and where it lands.

Is SECURE_CONTEXT ready for tenant-aware views?

Not for sole reliance, because it is in Preview. It is worth a prototype, though, because the problem it targets is real.

As described, SECURE_CONTEXT returns the value of a request-scoped secure context parameter that is supplied when the query runs. Logical views can then filter or evaluate data using context such as user attributes, department IDs or tenant identifiers.

Here is why that matters. Say an agency group keeps all client campaign data in one shared table. Each client should see only its own rows. Today you might build one view per client, or lean on a mapping table, or enforce it in the application layer. A view that reads a request-scoped value can express the rule once and apply it per request. Fewer views to maintain, and less drift between them.

The risk is the usual Preview risk. Behaviour can change before GA, and the notes don't list restrictions, so you shouldn't guess at them. Use these rules while it's in Preview:

• Prototype on a non-sensitive copy of the data.
• Write down the exact rows each test identity should and should not see, then check that result set against the expectation.
• Keep your current access controls in place. Add the new function alongside them, don't swap them out.
• Have a security owner review the view definitions before anything real depends on them.

That last point is not optional if the data is regulated. A filter that is almost right is worse than no filter, because people trust it.

What about data agent monitoring and the Data Engineering Agent?

Both are GA, and both matter mostly if you already use the agents.

The monitoring item lets you watch the performance, adoption, latency and costs of your data agents and their conversations through Google Cloud Observability in BigQuery. If a team has rolled out a conversational data agent to business users, this answers the questions that come up in the first month. Is anyone using it? Is it slow? What is it costing us? Without that view, you find out from a complaint or an invoice.

A simple habit helps. Pick the four dimensions the notes name, and review them on a fixed rhythm, say every two weeks, with whoever owns the agent. Low adoption with high cost is a signal to narrow the agent's scope or retrain the audience. High latency in a few conversations points to specific questions worth examining.

The Data Engineering Agent update is narrower. It now supports BigQuery AI, ML and object functions and statements in the Dataform SQLX pipelines it generates, and in agent instruction files. For an engineering team that already builds Dataform pipelines this way, it means the agent can produce pipelines that include those function types rather than steering around them. Review what it generates the way you'd review a colleague's pull request. Generated SQL still needs a human check on joins, filters and cost.

How should you roll this out, and who owns each step?

Don't try all five at once. You'll learn nothing from the results if you do. One change at a time, with an owner and a stop condition.

Step	Owner	Done when
Baseline query time and egress cost for one remote workload	Data platform lead	Two weeks of numbers are recorded
Trial Lakehouse caching on that catalog or connection	Data platform lead	Before and after comparison is written up
Run AI.PARSE_DOCUMENT on 20 real documents	Analytics engineer	Table and heading accuracy is counted
Prototype a SECURE_CONTEXT view on non-sensitive data	Data engineer with security reviewer	Test identities see exactly the expected rows
Review agent monitoring on a fixed rhythm	Agent owner	Adoption, latency and cost are reviewed every two weeks
Test a Data Engineering Agent pipeline using an AI function	Data engineer	Generated SQLX is reviewed and run in a dev project

Use a development project for everything above. Cost surprises are cheap to fix there and expensive in production.

What should you ignore or avoid?

Ignore the urge to enable caching everywhere because it sounds like a free speedup. Off by default is a hint that the benefit depends on your read pattern.

Don't treat a Preview feature as a finished control. SECURE_CONTEXT is interesting, and that is as far as the notes take it today.

Don't bulk-parse documents before you've inspected the output on messy samples. A function can run successfully and still return a table that is missing two columns.

Don't expect these notes to answer your pricing questions. They describe cost direction for caching, lower egress, and they say nothing about the price of the cached copies or the other features. Read Google's pricing pages for your region and contract.

And don't let the agent items drive a platform decision. If you have no data agents, the monitoring GA is background reading, not a project.

Where does an implementation partner help, and where can you go it alone?

You can do most of the trials yourself. Enabling caching on one catalog, parsing a handful of documents and watching a monitoring view are small, reversible jobs for any competent BigQuery team.

Help is worth paying for where the design is hard to reverse or the stakes are high. Three cases stand out. First, a multi-tenant access model built on SECURE_CONTEXT, once it matures, because a flaw there is a data exposure rather than a bug. Second, a cross-cloud architecture where you need to decide what stays remote, what gets cached and what gets copied, since that choice shapes your bill for years. Third, a document pipeline at volume, where parsing accuracy, review workflow and governance all have to work together.

If you're weighing outside support, start by reading about BigQuery implementation partners and use the comparison tool to line up candidates. When you're ready to ask for proposals, the RFP template helps you state the workloads, the baseline numbers and the success measure up front, so bids are comparable. More guidance on reading Google releases is on the MarTech Partners blog.

A good brief for any of these says what you measured, what you want to change, and how you'll know it worked. If a partner can't engage with that, keep looking.

Sources

• Google Cloud: BigQuery release notes, 2 November 2026 — Lakehouse caching GA, data agent monitoring GA, AI.PARSE_DOCUMENT GA, SECURE_CONTEXT Preview, Data Engineering Agent GA update.