Change how cm's priority is calculated #1040
Conversation
|
Looks good. 👍 |
|
@hjelmn With this priority scheme, one needs to specify which PML to use in order to be able to use the OFI MTL. Is there a way around this?
|
|
I didn't adjust ofi's priority. I didn't think it was considered ready for prime time. You can set using -mca mtl_ofi_priority 100 to force it to run. Should we increase the priority? |
|
I don't think we need to increase the priority yet. I was just hoping there would be a way to automagically select cm when an MTL is specified on the command line :-) |
|
You need to still force cm (or bump ofi's priority) because it shouldn't be selected before ob1 in production. Right now the component initialization code can not tell the difference between only one component being available and the user selecting a single component. |
|
I should not it worked before because someone added ofi as one of the components that bumped cm's priority to 30. |
This patch removes a priority check that disables cm if the previous pml had higher priority. The check was incorrect as coded and is unnecessary as we finalize all but one pml anyway. Fixes open-mpi#1035 Signed-off-by: Nathan Hjelm <[email protected]>
This commit removes code that checks the ob1 priority vs the previous priority. The previous priority is meaningless here and may only cause ob1 to disable itself when it shouldn't. Signed-off-by: Nathan Hjelm <[email protected]>
The mca_base_select function uses returned priorities to select the best component/module. This priority may be of use to the caller so pass that information back in an optional argument. If the priority is not needed pass NULL. Signed-off-by: Nathan Hjelm <[email protected]>
This commit changes the priority of mtl components to be relative to pml/ob1 and updates the mtl interface to expose this priority. cm now sets its own priority based on the priority of the selected mtl component. Signed-off-by: Nathan Hjelm <[email protected]>
|
So if I understand well, we should increase OFI MTL's priority to 30 in order to have what we had before the change, right? |
|
Sort of. We need to bump psm to 31 first to keep it ahead of ofi. I left ofi's relative priority alone here (10). We could also just set ofi between 20 (ob1) and 30 (psm). |
|
My understanding of the code is that the CM and PML will now inherit the priorities of their underlying modules (the CM the max of all available MTL, and OB1 the max of all available BTL), or is getting it's own priority form the corresponding MCA parameter. Thus if we have a purely priority-based solution we can make everything work. By default we can order the MTL in whatever order we consider adequate, but always higher than any BTL. Thus, as the OB1 will inherit the max priority of the BTLs, if any MTL is ready to work the CM (who will inherit this priority) will be picked up. Selecting the OB1 PML become as simple as increasing it's priority to something larger than any MTL. Sounds simple to implement, and pretty logical for the users. It does change the default (MTL instead of PML), but the change is sensible and desirable. |
|
@bosilca Pretty much. The only difference is the BTLs don't have a priority since there may be many transports in use. So with this commit ob1 still uses its own priority. In the future it might be useful to determine the priority for ob1 using the active btls. This way we could select ob1 when running only intra-node since vader is much faster than both mxm and psm. |
Change how cm's priority is calculated
…c-barrier-bug oshmem/scoll: Fix bug in basic/barrier algorithm
@bosilca Please comment. This is a companion to or replacement for #1039.