Skip to content

Instantly share code, notes, and snippets.

@benfalk

benfalk/super_batch.ex

Last active January 29, 2019 21:42
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save benfalk/a4b2c94693c8937ce700f65ea867b817 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save benfalk/a4b2c94693c8937ce700f65ea867b817 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
super_batch.ex
every_100 = Flow.Window.global |> Flow.Window.trigger_every(100)
API.container_id_stream
|> Stream.chunk_every(10, 10, [])
|> Flow.from_enumerable(min_demand: 1, max_demand: 20, stages: 4)
|> Flow.flat_map(fn container_ids ->
container_ids
|> API.get_containers_by_id
|> Enum.map(& &1.item_ids)
|> List.flatten
end)
|> Flow.partition(window: every_100)
|> Flow.reduce(fn -> [] end, &[&1 | &2])
|> Flow.on_trigger(& {[API.get_items_by_id(&1)], []})
@benfalk
Copy link
Author

benfalk commented Jan 29, 2019

baby-beard

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment