Get Ready to Pass the Associate-Developer-Apache-Spark exam with Databricks Latest Practice Exam [Q51-Q65]

June 27, 2022 Associate-Developer-Apache-Spark, Databricks 0 Comments

Get Ready to Pass the Associate-Developer-Apache-Spark exam with Databricks Latest Practice Exam [Q51-Q65]

Rate this post

Get Ready to Pass the Associate-Developer-Apache-Spark exam with Databricks Latest Practice Exam

Get Prepared for Your Associate-Developer-Apache-Spark Exam With Actual Databricks Study Guide!

NEW QUESTION 51
Which of the following describes a difference between Spark’s cluster and client execution modes?

In cluster mode, the cluster manager resides on a worker node, while it resides on an edge node in client mode.

In cluster mode, executor processes run on worker nodes, while they run on gateway nodes in client mode.

In cluster mode, the driver resides on a worker node, while it resides on an edge node in client mode.

In cluster mode, a gateway machine hosts the driver, while it is co-located with the executor in client mode.

In cluster mode, the Spark driver is not co-located with the cluster manager, while it is co-located in client mode.

NEW QUESTION 52
Which of the following is a viable way to improve Spark’s performance when dealing with large amounts of data, given that there is only a single application running on the cluster?

Increase values for the properties spark.default.parallelism and spark.sql.shuffle.partitions

Decrease values for the properties spark.default.parallelism and spark.sql.partitions

Increase values for the properties spark.sql.parallelism and spark.sql.partitions

Increase values for the properties spark.sql.parallelism and spark.sql.shuffle.partitions

Increase values for the properties spark.dynamicAllocation.maxExecutors, spark.default.parallelism, and spark.sql.shuffle.partitions

NEW QUESTION 53
Which of the following code blocks returns a DataFrame where columns predError and productId are removed from DataFrame transactionsDf?
Sample of DataFrame transactionsDf:
1.+————-+———+—–+——-+———+—-+
2.|transactionId|predError|value|storeId|productId|f |
3.+————-+———+—–+——-+———+—-+
4.|1 |3 |4 |25 |1 |null|
5.|2 |6 |7 |2 |2 |null|
6.|3 |3 |null |25 |3 |null|
7.+————-+———+—–+——-+———+—-+

transactionsDf.withColumnRemoved(“predError”, “productId”)

transactionsDf.drop([“predError”, “productId”, “associateId”])

transactionsDf.drop(“predError”, “productId”, “associateId”)

transactionsDf.dropColumns(“predError”, “productId”, “associateId”)

transactionsDf.drop(col(“predError”, “productId”))

NEW QUESTION 54
The code block displayed below contains an error. The code block should combine data from DataFrames itemsDf and transactionsDf, showing all rows of DataFrame itemsDf that have a matching value in column itemId with a value in column transactionsId of DataFrame transactionsDf. Find the error.
Code block:
itemsDf.join(itemsDf.itemId==transactionsDf.transactionId)

The join statement is incomplete.

The union method should be used instead of join.

The join method is inappropriate.

The merge method should be used instead of join.

The join expression is malformed.

NEW QUESTION 55
Which of the following describes the role of tasks in the Spark execution hierarchy?

Tasks are the smallest element in the execution hierarchy.

Within one task, the slots are the unit of work done for each partition of the data.

Tasks are the second-smallest element in the execution hierarchy.

Stages with narrow dependencies can be grouped into one task.

Tasks with wide dependencies can be grouped into one stage.

NEW QUESTION 56
Which of the following code blocks returns a copy of DataFrame transactionsDf that only includes columns transactionId, storeId, productId and f?
Sample of DataFrame transactionsDf:
1.+————-+———+—–+——-+———+—-+
2.|transactionId|predError|value|storeId|productId| f|
3.+————-+———+—–+——-+———+—-+
4.| 1| 3| 4| 25| 1|null|
5.| 2| 6| 7| 2| 2|null|
6.| 3| 3| null| 25| 3|null|
7.+————-+———+—–+——-+———+—-+

transactionsDf.drop(col(“value”), col(“predError”))

transactionsDf.drop(“predError”, “value”)

transactionsDf.drop(value, predError)

transactionsDf.drop([“predError”, “value”])

transactionsDf.drop([col(“predError”), col(“value”)])

NEW QUESTION 57
Which of the following code blocks concatenates rows of DataFrames transactionsDf and transactionsNewDf, omitting any duplicates?

transactionsDf.concat(transactionsNewDf).unique()

transactionsDf.union(transactionsNewDf).distinct()

spark.union(transactionsDf, transactionsNewDf).distinct()

transactionsDf.join(transactionsNewDf, how=”union”).distinct()

transactionsDf.union(transactionsNewDf).unique()

NEW QUESTION 58
Which of the following code blocks reads JSON file imports.json into a DataFrame?

spark.read().mode(“json”).path(“/FileStore/imports.json”)

spark.read.format(“json”).path(“/FileStore/imports.json”)

spark.read(“json”, “/FileStore/imports.json”)

spark.read.json(“/FileStore/imports.json”)

spark.read().json(“/FileStore/imports.json”)

NEW QUESTION 59
Which of the following code blocks immediately removes the previously cached DataFrame transactionsDf from memory and disk?

array_remove(transactionsDf, “*”)

transactionsDf.unpersist()
(Correct)

del transactionsDf

transactionsDf.clearCache()

transactionsDf.persist()

NEW QUESTION 60
The code block displayed below contains an error. The code block should return the average of rows in column value grouped by unique storeId. Find the error.
Code block:
transactionsDf.agg(“storeId”).avg(“value”)

Instead of avg(“value”), avg(col(“value”)) should be used.

The avg(“value”) should be specified as a second argument to agg() instead of being appended to it.

All column names should be wrapped in col() operators.

agg should be replaced by groupBy.

“storeId” and “value” should be swapped.

NEW QUESTION 61
Which of the following code blocks returns a DataFrame with approximately 1,000 rows from the 10,000-row DataFrame itemsDf, without any duplicates, returning the same rows even if the code block is run twice?

itemsDf.sampleBy(“row”, fractions={0: 0.1}, seed=82371)

itemsDf.sample(fraction=0.1, seed=87238)

itemsDf.sample(fraction=1000, seed=98263)

itemsDf.sample(withReplacement=True, fraction=0.1, seed=23536)

itemsDf.sample(fraction=0.1)

Explanation
itemsDf.sample(fraction=0.1, seed=87238)
Correct. If itemsDf has 10,000 rows, this code block returns about 1,000, since DataFrame.sample() is never guaranteed to return an exact amount of rows. To ensure you are not returning duplicates, you should leave the withReplacement parameter at False, which is the default. Since the question specifies that the same rows should be returned even if the code block is run twice, you need to specify a seed. The number passed in the seed does not matter as long as it is an integer.
itemsDf.sample(withReplacement=True, fraction=0.1, seed=23536)
Incorrect. While this code block fulfills almost all requirements, it may return duplicates. This is because withReplacement is set to True.
Here is how to understand what replacement means: Imagine you have a bucket of 10,000 numbered balls and you need to take 1,000 balls at random from the bucket (similar to the problem in the question). Now, if you would take those balls with replacement, you would take a ball, note its number, and put it back into the bucket, meaning the next time you take a ball from the bucket there would be a chance you could take the exact same ball again. If you took the balls without replacement, you would leave the ball outside the bucket and not put it back in as you take the next 999 balls.
itemsDf.sample(fraction=1000, seed=98263)
Wrong. The fraction parameter needs to have a value between 0 and 1. In this case, it should be 0.1, since
1,000/10,000 = 0.1.
itemsDf.sampleBy(“row”, fractions={0: 0.1}, seed=82371)
No, DataFrame.sampleBy() is meant for stratified sampling. This means that based on the values in a column in a DataFrame, you can draw a certain fraction of rows containing those values from the DataFrame (more details linked below). In the scenario at hand, sampleBy is not the right operator to use because you do not have any information about any column that the sampling should depend on.
itemsDf.sample(fraction=0.1)
Incorrect. This code block checks all the boxes except that it does not ensure that when you run it a second time, the exact same rows will be returned. In order to achieve this, you would have to specify a seed.
More info:
– pyspark.sql.DataFrame.sample – PySpark 3.1.2 documentation
– pyspark.sql.DataFrame.sampleBy – PySpark 3.1.2 documentation
– Types of Samplings in PySpark 3. The explanations of the sampling… | by Pinar Ersoy | Towards Data Science

NEW QUESTION 62
Which of the following describes a shuffle?

A shuffle is a process that is executed during a broadcast hash join.

A shuffle is a process that compares data across executors.

A shuffle is a process that compares data across partitions.

A shuffle is a Spark operation that results from DataFrame.coalesce().

A shuffle is a process that allocates partitions to executors.

NEW QUESTION 63
Which of the following describes characteristics of the Spark UI?

Via the Spark UI, workloads can be manually distributed across executors.

Via the Spark UI, stage execution speed can be modified.

The Scheduler tab shows how jobs that are run in parallel by multiple users are distributed across the cluster.

There is a place in the Spark UI that shows the property spark.executor.memory.

Some of the tabs in the Spark UI are named Jobs, Stages, Storage, DAGs, Executors, and SQL.

NEW QUESTION 64
Which of the following code blocks saves DataFrame transactionsDf in location /FileStore/transactions.csv as a CSV file and throws an error if a file already exists in the location?

transactionsDf.write.save(“/FileStore/transactions.csv”)

transactionsDf.write.format(“csv”).mode(“error”).path(“/FileStore/transactions.csv”)

transactionsDf.write.format(“csv”).mode(“ignore”).path(“/FileStore/transactions.csv”)

transactionsDf.write(“csv”).mode(“error”).save(“/FileStore/transactions.csv”)

transactionsDf.write.format(“csv”).mode(“error”).save(“/FileStore/transactions.csv”)

NEW QUESTION 65
Which of the following code blocks returns a single-column DataFrame showing the number of words in column supplier of DataFrame itemsDf?
Sample of DataFrame itemsDf:
1.+——+—————————–+——————-+
2.|itemId|attributes |supplier |
3.+——+—————————–+——————-+
4.|1 |[blue, winter, cozy] |Sports Company Inc.|
5.|2 |[red, summer, fresh, cooling]|YetiX |
6.|3 |[green, summer, travel] |Sports Company Inc.|
7.+——+—————————–+——————-+

itemsDf.split(“supplier”, ” “).count()

itemsDf.split(“supplier”, ” “).size()

itemsDf.select(word_count(“supplier”))

spark.select(size(split(col(supplier), ” “)))

itemsDf.select(size(split(“supplier”, ” “)))