Queue Key (Q_Key)
Contents
Since in RDMA any UD QP can send/receive a packet to/from any other UD QP in the subnet, there is a need for a protection mechanism to allow a UD QP to specify from which other UD QPs it wishes to get packets from. This mechanism is the Q_Key.
Q_Key is a 32 bits value used in UD QPs to validate that a remote sender has the right to access a local Receive Queue.
Using ibv_modify_qp() one can set Q_Key value to a specific QP context.
Q_Key in the Send Queue
When posting a SR to a UD QP, if the msb of the SR.Q_Key value is:
- Set - the Q_Key value that will be sent in the packet will be taken from the QP context.
- Cleared - the Q_Key value that will be sent in the packet will be taken from the SR.
Q_Key in the Receive Queue
Any incoming packet to a UD QP has a Q_Key value which will be compared with the Q_Key value in the receiver's Receive Queue. If they are:
- Match - the packet will be accepted
- Doesn't match - the packet will be silently dropped, the port_attr.qkey_viol_cntr will be increased if supported by the device (IBV_DEVICE_BAD_QKEY_CNTR is set in dev_cap.device_cap_flags). The receiver QP state won't be changed.
Q_Key values
As specified above, when posting a SR to a UD QP, if the msb of the Q_Key value in the SR is set, the Q_Key value that will be sent in the packet will be taken from the QP context. This provides a set of "privileged Q_Keys" and the only way for a QP to send or receive a privileged Q_Key is to set it in the QP context using ibv_create_qp().
Unprivileged Q_Keys
Any application can use any unprivileged Q_Key value in its QP context.
Privileged Q_Keys
Since calling ibv_create_qp() always involve executing code in the RDMA core (at the kernel level), a privileged code controls which applications can use the privileged Q_Keys for their UD QPs and enforce use of those values.
The values 0x80000000 - 0x8000FFFF are available for general use by applications.
The values 0x80010000 - 0x8FFFFFFF are reserved values and some specific values have a special meaning, for example: the value 0x80010000 is a well-known IB management Q_Key.
Share Our Posts
Share this post through social bookmarks.
Comments
Tell us what do you think.
Hi Dotan,
Will this Q_KEY not be used for Reliable datagram messages? Your article seems to suggest that this is only applicable for UD connections.
Thanks in advance!
Hi.
The Q_Key is used for all datagram messages; however, Reliable Datagram isn't supported (by the RDMA stack or by any RDMA device),
this is the reason that I completely ignore RD.
Thanks
Dotan
Hi Dotan,
It seems both P_Key and Q_Key are used to achieve isolation. What are differences between them? Or why can't we just use one of them?
Thank you very much!
Hi.
This is an interesting questions,
both of the features can be seen as mean of access control.
However, P_Keys are things that the Subnet Manager configures and enforce (it is like VLAN),
packet may be dropped by switch if it isn't part of that partition.
Q_Key is at the Queue Pair level, and is relevant only for Unreliable Datagram QPs
(which can get packets from any other QP in the subnet). The SW configures its value.
So, P_Key should prevent packets from reaching a QP which isn't member in that partition at all,
but Q_Key will prevent from QPs (in the same partition) which don't have the right Q_Key value.
Is this explanation clear?
Thanks
Dotan
From your explanation, the Hardware should have the QKey in one of its QP context registers. If yes, which verb programs the QKey to the HW.?
Hi.
ibv_modify_qp() configures the QP's Q_Key.
Thanks
Dotan