In a batch job that processes multiple 500 KB files in S3, the listing time exceeds 20 minutes due to a small-file problem. How would you fix that issue natively?

The small‑file problem inflates metadata operations and network overhead. I would first combine the small files into larger ones before processing. This can be done by running a MapReduce job or Spark job that reads the 500 KB files and writes them out as 100 MB Parquet files, preserving schema and partitioning. Alternatively, I would use AWS Glue or EMR to run a job that coalesces partitions. If the data is already in S3, I can use S3 DistCp or the AWS CLI to merge files locally and re‑upload. After consolidation, I would adjust the job’s input path to point to the new larger files, which reduces listing time and improves read throughput. Finally, I would enable S3 Transfer Acceleration or use S3 Select to further optimize data access.