ctietze,
@ctietze@mastodon.social avatar

async/await question:

This function produces one (1) array of results asynchronously:

func doTheThing() async -> [Result]

How would you write a (search/fetch/...) service that produces results in batches? E.g. if there are multiple data sources?

Am thinking of RxSwift's window.

Maybe return an array of the array of results, aka an AsyncSequence?

Or AsyncStream? 🤔

glotcha,
@glotcha@mastodon.social avatar

@ctietze there's an example in last years AsyncAlgorithms wwdc2022-110355 talk. The flow is AsyncStream (user input queries) of AsyncSequences (debounced queries) to AsyncChannel (results)

ctietze,
@ctietze@mastodon.social avatar

@glotcha AsyncChannel from the async-algorithms package sounds interesting. I have to read up on the reactice concepts more, though, to figure out if I need "back pressure" :)

glotcha,
@glotcha@mastodon.social avatar

@ctietze deciphering the docs here is very tricky. you say batches isn't that equivalent to "back-pressure"... meaning you want to ensure "that the production of values does not exceed the consumption of values from iteration" giving you an opportunity to process the multiple datasources for each search (iteration) before updating the UI state.

kaybutter,
@kaybutter@mastodon.social avatar

@ctietze An AsyncStream or some custom AsyncSequence is completely fine for this. Not sure why you’d think that’s only for long running tasks.
In my experience an AsyncSequence can be used for most things a publisher can be used for.
It’s not always easier though.

ctietze,
@ctietze@mastodon.social avatar

@kaybutter I'm not sure how to "switch to latest" with that just yet.

Say it's a user-interactive search. The text is debounced and then the search service is queried. That service pulls results from 5 data sources.

A callback, I'd call (up to) 5 times with the concatenated results, so in the end it'd be data source 1+2+3+4+5 combined.

Will task cancellation be enough to abort even after data source 1+2+3 have produces results if the user continues to type?

😵‍💫

kaybutter,
@kaybutter@mastodon.social avatar

@ctietze Yes, you probably need Apple's async-algorithms package for some these methods.

ctietze,
@ctietze@mastodon.social avatar

@kaybutter Will check that out, thanks!

ctietze,
@ctietze@mastodon.social avatar

AsyncStream<[Result]> 👎

That's for long-running streams, not for sequences that end after e.g. 4 results have been produced.

mattiem,
@mattiem@mastodon.social avatar

@ctietze I think the vision is exactly this + something from AsyncAlgorithms to combine them all together.

ctietze,
@ctietze@mastodon.social avatar

@mattiem Aaah, that's where it's hiding!

ctietze,
@ctietze@mastodon.social avatar

@mattiem @kaybutter There's no 'switch latest' in
https://github.com/apple/swift-async-algorithms

But it appears that when for-in stream consumption itself is in a Task, and that is being cancelled, the stream is terminated with cancellation as well.

Not sure if spawning 5 Tasks inside the stream to request data from all data sources, and then cancelling these tasks if the stream is cancelled is making this simpler to understand, though.

kaybutter,
@kaybutter@mastodon.social avatar

@ctietze @mattiem Oh, you're right. I guess I saw this pitch a while ago and I had hoped it would be in there by now 😄

https://forums.swift.org/t/prepitch-introduce-flat-maplatest-algorithms/60935/2

ctietze,
@ctietze@mastodon.social avatar

@kaybutter @mattiem Thanks for the link! The implementations shared there don't look that complicated, nice

uliwitness,
@uliwitness@chaos.social avatar

@ctietze This sounds like more of a job for a Combine Publisher than async/await?

ctietze,
@ctietze@mastodon.social avatar

@uliwitness I would be reaching for that out of habit, too. Sure that structured concurrency doesn't offer anything good? 🤔

uliwitness,
@uliwitness@chaos.social avatar

@ctietze Haven't really used it much, but yeah, some AsyncSequence or something else that you can await for item in doTheThing() { ... } would probably be the best, but I don’t remember the exact type one uses for that.

uliwitness,
@uliwitness@chaos.social avatar

@ctietze Yeah I think AsyncStream seems like it would fit your use case if you already have a callback-based API, going by the description here: https://www.andyibanez.com/posts/using-asyncsequence-in-swift/

But don’t use an array of arrays if the batches are just because that's how e.g. a paged request sends you the data. Do a one-dimensional array and just hand them to the AsyncStream in batches.

OTOH if you want to send categories of items (like groups in a table view) as batches, then of course the array of arrays makes sense.

uliwitness,
@uliwitness@chaos.social avatar

@ctietze Did I understand the problem right? You want to get list items as they arrive, not delay the notification until all have arrived, right? Otherwise, a colleague suggested withThrowingTaskGroup(of:), which is also a cool thing to have.

ctietze,
@ctietze@mastodon.social avatar

@uliwitness I fiddled with this a bit and actually combined both!

https://gist.github.com/DivineDominion/721334202833a2c0222f735dcf6a2d9c

This is pretty cool, but also a bit weird to look at :)

  • All
  • Subscribed
  • Moderated
  • Favorites
  • random
  • DreamBathrooms
  • mdbf
  • ngwrru68w68
  • magazineikmin
  • thenastyranch
  • rosin
  • khanakhh
  • osvaldo12
  • Youngstown
  • slotface
  • Durango
  • kavyap
  • InstantRegret
  • tacticalgear
  • anitta
  • ethstaker
  • provamag3
  • cisconetworking
  • tester
  • GTA5RPClips
  • cubers
  • everett
  • modclub
  • megavids
  • normalnudes
  • Leos
  • JUstTest
  • lostlight
  • All magazines