[Dev](EN) Timing issue in Pub/Sub and database write operation
Environment and Prerequisite
- Pub/Sub
- RDB
- Java
Background
- While processing list in parallel to publish Pub/Sub events, an issue occurred where the final state in the database was saved differently than intended.
- Data inconsistency is occurring even though all database write logs appear normal.
Issue
- Issues occurred in the logic that uses
parallelStream()to send bulk messages and batch update their statuses. - Although the system received a “Processing Succeeded (SUCCESS)” response from GCP Pub/Sub and updated the DB, the final result remained in the “Published (PUBLISHED)” state.
- The following pseudo-code illustrates the logic where the issue occurred.
// 1. Parallel event publishing (Main Thread)
jobList.parallelStream().forEach(job -> {
// Publishing Pub/Sub messages
publishEvent(job);
});
// ---------------------------------------------------------
// [Key Point] The SUCCESS response from the Pub/Sub subscrition arrives first!
// A separate subscription thread (Sub Thread) has already updated the DB to SUCCESS.
// ---------------------------------------------------------
// 2. Status update after publishing (Main Thread)
// This runs late and overwrites the DB with "PUBLISHED"
jobRepository.findAllById(ids).forEach(job -> {
// AS-IS: Unconditional overwrite without checking the current status
job.setStatus(PUBLISHED);
jobRepository.save(job); // Integrity error: SUCCESS reverts to PUBLISHED
});
Reason
Pub/Sub subscription response arriving faster than the DB write
- Expected Flow: Publish to Pub/Sub -> Change to “Published” (DB: PUBLISHED) -> Receive Subscription Message (DB: SUCCESS)
- Abnormal Flow: Publish to Pub/Sub -> Receive Subscription Message (DB: SUCCESS) -> Change to “Published” (DB: PUBLISHED)
Thoughts
I have encountered similar issues before regarding parallel or asynchronous processing and state updates ([Javascript](EN) async await is not work sequentially in forEach). It seems this is a recurring pattern that requires constant care during development.