BFloat16 data type naming

[Guobing-Chen]

In current code base, the BFloat16 data type is named as shxxxx (e.x.: shgemm), and related build flag as BUILD_HALF. Seems we simply take BF16 as half precision of float. This is not true as to the standard definition of IEEE. Half precsion should be FP16, which is different in format and content as compared with BFloat16. As for OpenBLAS, we may both support BFloat16 and and FP16 as valuable for different domains -- BFloat16 mostly valuable for Deep Learning and Machine Learning, while FP16 more valuable for traditional scientific computation and telecom processing.

Suggest to change the naming of data type and build flag to be bxxxx (e.x.: bgemm) and BUILD_BF16. While we can leave shxxxx for the real half precision data type -- FP16, or even make it as hxxxx (e.x.: hgemm). Whatever, use shxxxx and BUILD_HALF is quite confusing to community, every other math libs like Eigen/oneDNN (previously mkldnn) use keywords like bf or bf16 or so.

I can submit PR to make this change if we are OK on this.

[RajalakshmiSR]

The first 's' is for the result (single-precision). If b is for the input (bf16), bgemm would imply that both input and output are bfloat16, which is not the case. So 'sbgemm' would explain it better.

[Guobing-Chen]

Good point! 'sb' prefix is good to me also as specific to bfloat16.

[martin-frbg]

I remember suggesting to @edelsohn that function names be coordinated with other interested parties before the code lands,
but I have to admit that seeing the actual implementation might be a prerequisite. Also I have no clear idea what the appropriate panel for deciding naming and API for BLAS extensions would be these days ?

[Guobing-Chen]

I remember suggesting to @edelsohn that function names be coordinated with other interested parties before the code lands,
but I have to admit that seeing the actual implementation might be a prerequisite. Also I have no clear idea what the appropriate panel for deciding naming and API for BLAS extensions would be these days ?

Do you mean naming thing and new APIs should be discussed in a specific panel? If Yes, can I attend? Besides this naming, I also have some new APIs (extensions) to suggest to add.

[martin-frbg]

TBH I haven't the froggiest idea how this is currently handled, if at all. Back in the early days of BLAS and LAPACK there must have been enough of a working group to leave meeting notes and whitepapers for posterity, but as I came into my maintainer role purely from a user background as a computational chemist I do not know if any such coordination is still taking place.

[Guobing-Chen]

Emmm... then can we make it simple to just add a note in the release note, considering that this BF16 based shgemm is just added recently that suppose very little users will be impacted.

Also a side question may be silly one -- I don't see this shgemm API been added in cblas.h, then how a user will use it normally?

[martin-frbg]

Good point - I am old enough to consider FORTRAN "normal use" though 😄

[Guobing-Chen]

Definitely you run so quickly to that far away LOL

[martin-frbg]

Still curious - is there any coordination on BLAS/LAPACK extension APIs still happening among major interested parties @langou @cparrott73 or does everybody get to name their pet function as they like ?

[Guobing-Chen]

I created a PR about the change of the naming based on this proposal and RajalakshmiSR's comment. We can update the PR more if we have more idea on the naming/prefix.

[edelsohn]

The naming was coordinated with and approved by Jack Dongarra, who is the leader of NETLIB. OpenBLAS cannot start inventing names.

[martin-frbg]

Thx. I vaguely remembered your stated intention but not any outcome, and did not see it mentioned at netlib.I guess OpenBLAS should evolve its own documentation eventually - added the information to the "OpenBLAS extensions" page in the wiki for now.

[brada4]

IMO extensions should be configurable out so that people dont get into thinking they somehow are part of BLAS and build eventually broken executables.