metal : add memory pool for temp allocs

[ggerganov]

ref ggml-org/ggml#1152 (comment)

The goal is to introduce a mechanism that allows to allocate temporary buffers in the Metal backend that can be used to store intermediate results. This is needed for some composite operations (like convolution represented by im2col + mul_mat) or for rearranging or padding data on the fly. This is similar to the ggml_cuda_pool_alloc functionality in the CUDA backend.

For testing, currently using the SOFT_MAX operation by introducing an intermediate step of copying the input data to an intermediate buffer and then running the softmax kernel on that intermediate buffer (instead of on the input one).

make -j && MTL_DEBUG_LAYER=1 ./bin/test-backend-ops -b Metal -o SOFT_MAX

TODO:

• Figure out how to create MTLHeap and allocate buffers from it
• How to release the buffers
• Create per-command-buffer heaps
• How to dynamically resize the heap based on the memory need of the graph
• Start using MTLHeapTypePlacement to be able to reuse heap memory from previous nodes
• Un-encode the failed encoder - how? Maybe recreate the command buffer?
• Check for memory leaks
• Try to allocate the MTLHeaps dynamically in order to avoid the extra loop over the nodes.
• Add comments

Next PRs:

• Use this new functionality to add F16 x F16 MUL_MAT support by casting src1 from F32 to F16
• Implement im2col + mul_mat for GGML_OP_CONV_XXX