Persisting Data in a Twisted App -

i'm trying understand how persist data in twisted application. let's i've decided write twisted server that:

1. accepts inbound smtp requests
2. sends message 3rd party system modification
3. relays modified message destination

a typical twisted tutorial have build app using deferreds , callbacks, roughly:

• a factory handles inbound requests
• each time full email received call sent remote message processor, returning deferred
• add errback substitutes original message if goes wrong in modify call.
• add callback send message on recipient, again returns deferred.
• a real server add/include additional call/errbacks retry or notify sender or whatnot. again simplicity, assume consider acceptable amount of effort , log errors.

of course, persists no data in event of crash/restart/something else. solution involves 3rd party persistent datastore (rabbitmq mentioned) , come dozen random ways achieve outcome.

however, imagine there few approaches work best in twisted app. like? how store (and restore in event of crash) in-process messages?

if found question, know twisted event-based. sounds simple, "hardest" part of answer persistence platform generating events need when need them. naturally, can persist data in db or message queue, platforms don't naturally generate events. example:

• zeromq has (or @ least had) no callback new data. it's relatively poor @ persistence.

in other cases, events easy reliability problem:

• pgsql can configured generate events using triggers, they're one-time things can't resume incomplete events

the light @ end of tunnel seems rabbitmq.

• rabbitmq can persist message in database survive crash
• we can use acknowledgements on both legs (incoming smtp rabbitmq , rabbitmq outgoing smtp) ensure application reliable. importantly, rabbitmq supports acknowledgements.
• finally, several of rabbitmq clients provide full asynchronous support (see example pika, txampq, , puka)

it's enough our purposes rabbitmq client provides event-based interface.

at more theoretical level, however, need not case. in fact, despite "notification" issue above, zeromq has event-based client. if our software elegantly event-based, run systems aren't. in these cases, have no choice fall on polling. in principle, if not in practice, query message provider messages. when exhaust current queue (and if there no messages), use calllater command check again in future. may feel anti-pattern, it's (to best of knowledge anyway) right way handle particular case.