Hmm, I'm a bit baffled by this last post in this post series.
Although not explicitly stated I believe the original question was how to achieve caching of expensive calculation results with low memory impact (MemoryCache) and low allocation count (ArrayPool) without having additional bugs.
Maybe I was wrong in my expectation?
This post brings up a wrapper object (ReturnBuffer) which needs to be allocated in memory (so we go up in allocation) and this object will also get placed into finalizer queue by GC to ensure release of memory.
As the entire object that we are holding is actually just 32 bytes we could have just returned the copy of the original array and I assume that the performance (speed/memory) would be roughly equivalent or even in favor of the simple copy.
Frankly at this point I believe someone else wrote the post and not Oren:
- There seems to be no purpose to_joinLifetimes ('...that was added to the runtime to enable dynamic properties on objects. Basically, it creates a table that you can lookup by an object to get a key' --> I can not see Oren writing this)
- Caller which gets back the ReturnBuffer can change the content of the array
- Caller can also take direct reference to the byte[] from the ReturnBuffer.Buffer and keep it around longer than it is 'alive'
- Caller can swap the ReturnBuffer.Buffer with some other byte[] array

Maybe I missed something completely here :(

I don't think that this class is exposed to the caller. It looks like it is only used internally to avoid "use after free" bug. But to be honest I also think this looks like an overkill unless you know (from usage data) that this will be called a lot of times, so a lot of copies are avoided. But what about ArrayPool exhaustion that was mentioned on previous post?

Have you considered not using ArrayPool at all in this scenario? I don't know how long the cache will keep the array, or how large they'll be, but it seems to me that renting a pooled array for a long, maybe unknown time could have negative effects on the ArrayPool. Won't it have to re-allocate new arrays if it runs out of rented ones? Wouldn't it be pointless then to return a rented array that has been held so long that the pool "forgot" about it?

Would it perhaps be acceptable to just allocate an array in this case, since it will be cached and re-used anyway? This code is definitely not simple anymore and there are many ways someone might introduce subtle bugs accidentally. If it is performance critical, then of course. If I remember correctly from the original article, this is "new" code though, so who knows if it's actually needed, or perhaps premature optimization.

This is basically a technique to add a finalizer to an object that you don't control.

This is a very nice trick, so the solution is clearly interesting. However, there is now one allocation for every cached buffer, so it might be much simpler to just allocate the initial buffer. Also, if you know that the number of cached files will be very small, you can use a dedicated pool for the buffers, and never returns them.

We also expect that object to be collected quite quickly

Well, if you write "no-allocation" code, I suspect that your objects are not going to be collected :)

The pool doesn't remember the arrays that it rented out. There is no tracking of that at all.

I meant it conceptually "forgot", which is why I put quotation marks around it, but that obviously wasn't clear. What I mean is, if we rent all arrays that a pool has to offer, it will have to allocate new ones when further requests are made. Assume a pool can hold n buffers and we rent n, then we rent another n which have to be allocated anew. Then we return the first batch of n buffers, then the second batch. Now the pool has 2*n buffers it could use or free, but the point is that the second batch had to be allocated and essentially the pool might as well not have existed.

If the problem as I have outlined it exists, isn't there a risk then that your cache could cause this scenario? By holding buffers obtained from a pool for such a long time (and assuming that you make heavy use of the pool, so that it runs the risk of running out of buffers), then you could cause the very thing you try to avoid: allocating large arrays, in random places (whatever requests a buffer from the pool once it is empty).

This solution depends on implementation details of the ArrayPool. If it holds on to a reference for the buffer this solution will leak memory as the ReturnBuffer is never collected. I also assume you did measure that sparing the allocation of 32 bytes is worth it, as Finalizers & WeakReferences have their own overhead for the GC.