Dfyjkt

Issue:
We currently have two azure consumption plan functions, each receiving service bus queue messages as input.
The first functions call SQL Azure with a stored proc, gets 500k+ records back, saves those records in batches of a 100 to Azure table storage with each batch having a unique partition key. After that's done it then creates a new queue message for next function to read batch and process it.

Everything works fine when the second function is not running warm and still needs to warm up. If the second function is running in memory, and it receives the queue message, we do a partition key lookup against the table storage, and sometimes it seems the data coming back is empty.

Code that inserts batches into table storage:

 foreach (var entry in partitionKeyGroupinng)
 
 var operation = new TableBatchOperation();
 entry.ToList().ForEach(operation.Insert);

 if (operation.Any())
 
 await CloudTable.ExecuteBatchAsync(operation);
 
 

This is within an async task function in a shared assembly referenced by all functions.

Code to read out from table storage as partition key lookup:

TableContinuationToken continuationToken = null;

 var query = BuildQuery(partitionKey);

 var allItems = new List<T>();
 do
 
 var items = await CloudTable.ExecuteQuerySegmentedAsync(query, continuationToken);
 continuationToken = items.ContinuationToken;
 allItems.AddRange(items);
 while (continuationToken != null);

 return allItems;

Code that calls that to lookup by partition key:

 var batchedNotifications = await _tableStorageOperations.GetByPartitionKeyAsync($"trackingId.ToString()_batchNumber");

I reckon its to do with the batch still being written and available to other clients but don't know if that's the case? What would be the best way to handle this with the function processing and eventual consistency?

I have disabled the following on table client:

 tableServicePoint.UseNagleAlgorithm = false; 
 tableServicePoint.Expect100Continue = false;
 tableServicePoint.ConnectionLimit = 300;

If I also look up that same partition key in storage explorer as the event happens, I can see the batch so it returns values? I thought to make use of EGT with the batching would ensure this is written and available as soon as possible, because the method async Task WriteBatch shouldn't finish before it has finished writing the batch, however, don't know how long the back of table storage takes to write that to a physical partition and then make it available. I have also batched all the service bus queue messages up before sending them to add some delay to the second function.

Question:
How do we deal with this delay in accessing these records out of table storage between two functions using service bus queues?