Databricks Certified-Data-Engineer-Professional real exam prep : Databricks Certified Data Engineer Professional

An upstream source writes Parquet data as hourly batches to directories named with the current date. A nightly batch job runs the following code to ingest all data from the previous day as indicated by the date variable:

Assume that the fields customer_id and order_id serve as a composite key to uniquely identify each order.
If the upstream system is known to occasionally produce duplicate entries for a single order hours apart, which statement is correct?

• A. Each write to the orders table will only contain unique records; if existing records with the same key are present in the target table, the operation will tail.
• B. Each write to the orders table will only contain unique records; if existing records with the same key are present in the target table, these records will be overwritten.
• C. Each write to the orders table will only contain unique records, and only those records without duplicates in the target table will be written.
• D. Each write to the orders table will run deduplication over the union of new and existing records, ensuring no duplicate records are present.
• E. Each write to the orders table will only contain unique records, but newly written records may have duplicates already present in the target table.

The data engineering team maintains the following code:

Assuming that this code produces logically correct results and the data in the source tables has been de-duplicated and validated, which statement describes what will occur when this code is executed?

• A. An incremental job will detect if new rows have been written to any of the source tables; if new rows are detected, all results will be recalculated and used to overwrite the enriched_itemized_orders_by_account table.
• B. The enriched_itemized_orders_by_account table will be overwritten using the current valid version of data in each of the three tables referenced in the join logic.
• C. No computation will occur until enriched_itemized_orders_by_account is queried; upon query materialization, results will be calculated using the current valid version of data in each of the three tables referenced in the join logic.
• D. A batch job will update the enriched_itemized_orders_by_account table, replacing only those rows that have different values than the current version of the table, using accountID as the primary key.
• E. An incremental job will leverage information in the state store to identify unjoined rows in the source tables and write these rows to the enriched_iteinized_orders_by_account table.

Which statement regarding stream-static joins and static Delta tables is correct?

• A. Each microbatch of a stream-static join will use the most recent version of the static Delta table as of each microbatch.
• B. Each microbatch of a stream-static join will use the most recent version of the static Delta table as of the job's initialization.
• C. Stream-static joins cannot use static Delta tables because of consistency issues.
• D. The checkpoint directory will be used to track state information for the unique keys present in the join.
• E. The checkpoint directory will be used to track updates to the static Delta table.

A data engineer is tasked with ensuring that a Delta table in Databricks continuously retains deleted files for 15 days (instead of the default 7 days), in order to permanently comply with the organization's data retention policy. Which code snippet correctly sets this retention period for deleted files?

• A. from delta.tables import *
  deltaTable = DeltaTable.forPath(spark, "/mnt/data/my_table")
  deltaTable.deletedFileRetentionDuration = "interval 15 days"
• B. spark.sql("ALTER TABLE my_table SET TBLPROPERTIES
  ('delta.deletedFileRetentionDuration' = 'interval 15 days')")
• C. spark.sql("VACUUM my_table RETAIN 15 HOURS")
• D. spark.conf.set("spark.databricks.delta.deletedFileRetentionDuration", "15 days")

A data team is automating a daily multi-task ETL pipeline in Databricks. The pipeline includes a notebook for ingesting raw data, a Python wheel task for data transformation, and a SQL query to update aggregates. They want to trigger the pipeline programmatically and see previous runs in the GUI. They need to ensure tasks are retried on failure and stakeholders are notified by email if any task fails. Which two approaches will meet these requirements? (Choose two.)

• A. Use the REST API endpoint /jobs/runs/submit to trigger each task individually as separate job runs and implement retries using custom logic in the orchestrator.
• B. Create a multi-task job using the UI, Databricks Asset Bundles (DABs), or the Jobs REST API (/jobs/create) with notebook, Python wheel, and SQL tasks. Configure task-level retries and email notifications in the job definition.
• C. Create a single orchestrator notebook that calls each step with dbutils.notebook.run(), defining a job for that notebook and configuring retries and notifications at the notebook level.
• D. Trigger the job programmatically using the Databricks Jobs REST API (/jobs/run-now), the CLI (databricks jobs run-now), or one of the Databricks SDKs.
• E. Use Databricks Asset Bundles (DABs) to deploy the workflow, then trigger individual tasks directly by referencing each task's notebook or script path in the workspace.