Need PySpark code to compute acceptance rate per date.

Question

Assisting AI · Accepted Answer

Here is a PySpark snippet that calculates the acceptance rate per date:

```python
from pyspark.sql import functions as F

# Filter and aggregate sent actions
sent = df.filter(F.col("action") == "sent")
    .groupBy("date")
    .agg(F.count("*" ).alias("sent_cnt"))

# Filter and aggregate accepted actions
accepted = df.filter(F.col("action") == "accepted")
    .groupBy("date")
    .agg(F.count("*" ).alias("accepted_cnt"))

# Join aggregates and compute rate
rate_df = sent.join(accepted, "date", "inner")
    .withColumn("acceptance_rate", (F.col("accepted_cnt") / F.col("sent_cnt")) * 100)
    .select("date", "acceptance_rate")

rate_df.show()
```

This code performs two groupBy operations, joins the results on date, and calculates the acceptance percentage. The overall time complexity is linear in the number of rows, with an additional cost for the join proportional to the number of distinct dates.

Need PySpark code to compute acceptance rate per date.

💡 Model Answer

🎤 Get questions like this answered in real-time