Metadata / Data Quality / Platforms
AWS sets out pattern to score SageMaker catalogue assets against governance rules using metadata export
A sample pipeline joins the SageMaker asset metadata export with form, glossary, data quality and subscription data to give each asset a weighted daily compliance score, and flags heavily used, poorly governed data.
AWS has published a reference pattern for Amazon SageMaker Unified Studio that turns catalogue metadata into a daily, per-asset compliance score. It builds on the asset metadata export, which writes catalogue inventory to an Apache Iceberg table in S3 Tables once a day.
A Lambda function joins the export with API data on attached metadata forms, glossary terms, data quality results and active subscriptions. Required forms combine SageMaker’s own enforcement rules with organisation-defined rules held in an S3 policy file. The example weights are form coverage 50%, data quality score 20%, and 10% each for data quality rule coverage, glossary presence and a description; AWS stresses these are configurable.
Scores feed Athena queries, EventBridge alerts and a Quick Sight dashboard. A risk matrix marks assets scoring below 50 with two or more subscribing projects as needing action now. Code is in an aws-samples repository.
Editor’s note: This article relies solely on AWS’s own blog post and product documentation.
Why it matters
The pattern makes metadata completeness measurable: owners can list assets missing required forms, stewards can find assets with no glossary terms, and CDOs can track whether compliance improves after new rules are published. Weighting findings by the number of consuming projects is a practical way to prioritise remediation backlogs. Teams on other catalogues can borrow the idea of combining platform-enforced and organisation-defined metadata requirements into one policy, but should set their own weights and thresholds rather than adopting the example scoring.
Status: Published. Sample code and walkthrough; scoring model described as a starting point
Sources
Was this useful?
Thanks. What was the problem?
Thank you — this helps us choose what to cover. If you spotted an error, please tell us what it is.
Prepared with AI assistance and reviewed by the editor before publication. Spotted an error? Tell us — see our editorial policy and methodology.
Governance areas
Who should care
- Data stewards
- Data owners
- CDO
- Data platform teams