The back end design of a social sharing system. Users post pictures or videos to social sites, often a time consuming operation. To avoid blocking the system, we use an asynchronous system design. The back end consists of several parts.
One post service. Handles API requests from the front end, including job submissions and job status queries.
Two post job consumer. Processes job messages, publishes content to social sites and updates job status.
Three message queue MQ. Manages the asynchronous handling of jobs.
Four database DB. Stores job details and their processing status.
Here’s how it operates.
One user initiate post request from the front end.
Two post service receives the request, converts it into a job, stores it in the database, and sends a job message to the MQ.
Three at this point, the job has been successfully submitted and the post service returns a job ID to the front end.
Since inserting data into the database and MQ involves lightweight operations, the job submission process is quick and does not block the front end.
Four the front end, having received a job ID, periodically polls the post service to check the job’s execution status.
Five if the job is queued or in progress, the front end receives an in queue or in progress status.
Six concurrently, post job consumer picks up the job from the MQ, begins publishing to the social site and updates the final status, success or failure, in the database.