Production-first Integrations
Understand the recommended event-driven architecture for integrating bem into your product for maximum scalability and reliability using asynchronous webhooks.
While you can always poll the bem API to fetch results, in a production environment, bem works best when fully integrated into your product using an event-driven, asynchronous architecture. This pattern is highly scalable, resilient, and ensures your application receives results with minimal latency as soon as they are ready.
The core principle is simple: your application calls a bem Function, bem does its work in the background, and then notifies your application via a webhook Subscription when the work is complete.
Recommended Integration Flow
The diagram below illustrates the ideal data flow between your application and bem.
Breakdown of the Flow
- Call
bem****: Your application initiates the process by making aPOSTrequest to a specificbemFunction (e.g.,POST /v2/functions/enterprise-invoice-processor/call). You send the data to be processed and a uniquereferenceIDthat you can use to track the job in your own system. - Acknowledge:
bemimmediately acknowledges that the request has been received and queued for processing by returning a200 OKresponse. Your application is now free to continue its own work without waiting. - Asynchronous Processing:
bemprocesses the data in the background. This could take anywhere from a few seconds to several minutes depending on the complexity of the task and the size of the input. - Webhook Event: Once processing is complete,
bemsends aPOSTrequest to the webhook URL you configured in your Subscription. This request contains the full event payload, including the originalreferenceIDand the structured data result. Your webhook endpoint can be implemented in various ways depending on your existing architecture:- Serverless Functions (Recommended): The most common and scalable approach is to use a serverless function fronted by an API Gateway. This is cost-effective, scales automatically, and is easy to maintain. Examples include AWS Lambda, Google Cloud Functions, or Azure Functions.
- Containerized Microservice: You can deploy a small, dedicated service in a container orchestrator like Kubernetes, Amazon ECS, or Google Cloud Run. This is a good choice if you have an existing container-based infrastructure.
- iPaaS Platforms: Many enterprises use Integration Platform as a Service (iPaaS) tools to manage workflows between SaaS applications. Your endpoint could be a workflow trigger in a platform like Workato, MuleSoft, or Zapier.
- Monolithic Application Endpoint: If you run a traditional monolithic application, you can simply expose a new
POSTendpoint (e.g.,/api/webhooks/bem) within your existing codebase to receive the events.
- Acknowledge Receipt: Your webhook endpoint should be designed to quickly receive the data, perhaps place it onto an internal queue (like SQS or RabbitMQ) for processing, and immediately return a
200 OKstatus code tobemto confirm receipt. - Process and Store: After acknowledging the webhook, a separate worker process can then perform your own business logic, such as validating the data, updating a record in your database, or triggering downstream workflows.
This asynchronous, event-driven pattern ensures that your application remains responsive and is not blocked waiting for long-running processes, making it the ideal architecture for building scalable, production-grade products with bem.