Solicited Event is a mechanism for the requester side to send a message that will generate a special (i.e. solicited) event on the responder side when the message is received. The event will be generated for the message when a Work Completion is added to the Completion Queue at the responder side (in the Receive Queue), therefore it is only valid for the Send, Send with immediate and RDMA Write with immediate operations (since only those operations generate a Work Completion at the responder side).

When to use Solicited Events?

If one wishes to decrease the CPU consumption and instead of working with regular Work Completion events, i.e. wait for the next Work Completion that added to the Completion Queue, and work with the Work Completions that the sender side mark them as "special" the Solicited Event mechanism is the answer.

Using Solicited events

On the sender side

One should call ibv_post_send(), he needs to set the IBV_SEND_SOLICITED bit in send_wr.send_flags. This is relevant only to the operations which were mentioned above.

On the receiver side

One should call ibv_req_notify_cq() on a Receive Completion Queue which may get an incoming message with a Solicited event; the value of solicited_only should be non-zero.

This will cause ibv_get_cq_event() to be blocked until one of the following will occur:

• the next Work Completion for an incoming message with a solicited event indicator will be added to the Completion Queue. This may happen at most once and if there is a need to wait again for the next solicited event, one may need to call ibv_req_notify_cq() again.
• Any Work Completion with error is added to the Completion Queue (either if it came from the Send or the Receive Queues)

For more information, one should read the posts about the mentioned above verbs.

Written by: Dotan Barak on May 27, 2014.on February 20, 2015.