Databricks Certified Associate Developer for Apache Spark 3.0 Exam (Associate-Developer-Apache-Spark) Free Practice Test

Question 1

Which of the following code blocks writes DataFrame itemsDf to disk at storage location filePath, making sure to substitute any existing data at that location?

A. itemsDf.write(filePath, mode="overwrite")

B. itemsDf.write().parquet(filePath, mode="overwrite")

C. itemsDf.write.mode("overwrite").path(filePath)

D. itemsDf.write.mode("overwrite").parquet(filePath)

E. itemsDf.write.option("parquet").mode("overwrite").path(filePath)

Correct Answer: D

Explanation: Only visible for TestSimulate members. You can sign-up / login (it's free).

Question 2

Which of the following describes a way for resizing a DataFrame from 16 to 8 partitions in the most efficient way?

A. Use a narrow transformation to reduce the number of partitions.

B. Use operation DataFrame.coalesce(8) to fully shuffle the DataFrame and reduce the number of partitions.

C. Use a wide transformation to reduce the number of partitions.
Use operation DataFrame.coalesce(0.5) to halve the number of partitions in the DataFrame.

D. Use operation DataFrame.repartition(8) to shuffle the DataFrame and reduce the number of partitions.

Correct Answer: A

Explanation: Only visible for TestSimulate members. You can sign-up / login (it's free).

Question 3

The code block shown below should return a copy of DataFrame transactionsDf with an added column cos.
This column should have the values in column value converted to degrees and having the cosine of those converted values taken, rounded to two decimals. Choose the answer that correctly fills the blanks in the code block to accomplish this.
Code block:
transactionsDf.__1__(__2__, round(__3__(__4__(__5__)),2))

A. 1. withColumn
2. col("cos")
3. cos
4. degrees
5. transactionsDf.value

B. 1. withColumn
2. "cos"
3. cos
4. degrees
5. transactionsDf.value

C. 1. withColumnRenamed
2. "cos"
3. cos
4. degrees
5. "transactionsDf.value"

D. 1. withColumn
2. col("cos")
3. cos
4. degrees
5. col("value")
E
. 1. withColumn
2. "cos"
3. degrees
4. cos
5. col("value")

Correct Answer: B

Explanation: Only visible for TestSimulate members. You can sign-up / login (it's free).

Question 4

Which of the following code blocks performs an inner join of DataFrames transactionsDf and itemsDf on columns productId and itemId, respectively, excluding columns value and storeId from DataFrame transactionsDf and column attributes from DataFrame itemsDf?

A. 1.transactionsDf.createOrReplaceTempView('transactionsDf')
2.itemsDf.createOrReplaceTempView('itemsDf')
3.
4.spark.sql("SELECT -value, -storeId FROM transactionsDf INNER JOIN itemsDf ON productId==itemId").drop("attributes")

B. transactionsDf.drop('value', 'storeId').join(itemsDf.select('attributes'), transactionsDf.productId==itemsDf.itemId)

C. transactionsDf.drop("value", "storeId").join(itemsDf.drop("attributes"),
"transactionsDf.productId==itemsDf.itemId")

D. 1.transactionsDf.createOrReplaceTempView('transactionsDf')
2.itemsDf.createOrReplaceTempView('itemsDf')
3.
4.statement = """
5.SELECT * FROM transactionsDf
6.INNER JOIN itemsDf
7.ON transactionsDf.productId==itemsDf.itemId
8."""
9.spark.sql(statement).drop("value", "storeId", "attributes")

E. 1.transactionsDf \
2. .drop(col('value'), col('storeId')) \
3. .join(itemsDf.drop(col('attributes')), col('productId')==col('itemId'))

Correct Answer: D

Explanation: Only visible for TestSimulate members. You can sign-up / login (it's free).

Question 5

The code block displayed below contains one or more errors. The code block should load parquet files at location filePath into a DataFrame, only loading those files that have been modified before
2029-03-20 05:44:46. Spark should enforce a schema according to the schema shown below. Find the error.
Schema:
1.root
2. |-- itemId: integer (nullable = true)
3. |-- attributes: array (nullable = true)
4. | |-- element: string (containsNull = true)
5. |-- supplier: string (nullable = true)
Code block:
1.schema = StructType([
2. StructType("itemId", IntegerType(), True),
3. StructType("attributes", ArrayType(StringType(), True), True),
4. StructType("supplier", StringType(), True)
5.])
6.
7.spark.read.options("modifiedBefore", "2029-03-20T05:44:46").schema(schema).load(filePath)

A. The attributes array is specified incorrectly, Spark cannot identify the file format, and the syntax of the call to Spark's DataFrameReader is incorrect.

B. Columns in the schema definition use the wrong object type, the modification date threshold is specified incorrectly, and Spark cannot identify the file format.

C. Columns in the schema definition use the wrong object type and the syntax of the call to Spark's DataFrameReader is incorrect.

D. Columns in the schema are unable to handle empty values and the modification date threshold is specified incorrectly.

E. The data type of the schema is incompatible with the schema() operator and the modification date threshold is specified incorrectly.

Correct Answer: B

Explanation: Only visible for TestSimulate members. You can sign-up / login (it's free).

Question 6

The code block displayed below contains an error. The code block should return a new DataFrame that only contains rows from DataFrame transactionsDf in which the value in column predError is at least 5. Find the error.
Code block:
transactionsDf.where("col(predError) >= 5")

A. Instead of where(), filter() should be used.

B. The argument to the where method cannot be a string.

C. Instead of >=, the SQL operator GEQ should be used.

D. The expression returns the original DataFrame transactionsDf and not a new DataFrame. To avoid this, the code block should be transactionsDf.toNewDataFrame().where("col(predError) >= 5").

E. The argument to the where method should be "predError >= 5".

Correct Answer: E

Explanation: Only visible for TestSimulate members. You can sign-up / login (it's free).

Question 7

Which of the following statements about Spark's DataFrames is incorrect?

A. The data in DataFrames may be split into multiple chunks.

B. Spark's DataFrames are equal to Python's DataFrames.

C. RDDs are at the core of DataFrames.

D. Spark's DataFrames are immutable.

E. Data in DataFrames is organized into named columns.

Correct Answer: B

Explanation: Only visible for TestSimulate members. You can sign-up / login (it's free).

Question 8

Which of the following code blocks returns all unique values across all values in columns value and productId in DataFrame transactionsDf in a one-column DataFrame?

A. transactionsDf.select('value').union(transactionsDf.select('productId')).distinct()

B. transactionsDf.select(col('value'), col('productId')).agg({'*': 'count'})

C. transactionsDf.agg({'value': 'collect_set', 'productId': 'collect_set'})

D. transactionsDf.select('value', 'productId').distinct()

E. tranactionsDf.select('value').join(transactionsDf.select('productId'), col('value')==col('productId'),
'outer')

Correct Answer: A

Explanation: Only visible for TestSimulate members. You can sign-up / login (it's free).

Question 9

Which of the following options describes the responsibility of the executors in Spark?

A. The executors accept jobs from the driver, analyze those jobs, and return results to the driver.

B. The executors accept tasks from the cluster manager, execute those tasks, and return results to the driver.

C. The executors accept jobs from the driver, plan those jobs, and return results to the cluster manager.

D. The executors accept tasks from the driver, execute those tasks, and return results to the driver.

E. The executors accept tasks from the driver, execute those tasks, and return results to the cluster manager.

Correct Answer: D

Explanation: Only visible for TestSimulate members. You can sign-up / login (it's free).

Welcome to TestSimulate

Databricks Certified Associate Developer for Apache Spark 3.0 (Associate-Developer-Apache-Spark) Free Practice Test