Skip to content

GitLab

Explore

Sign in

Primary navigation

Project

S

scalable-c4-preprocessing
- Activity
- Members
- Labels
- Issues
- Issue boards
- Milestones
- Wiki
- Requirements
- Terraform modules
- Incidents
- Service Desk

Snippets Groups Projects

defc1823

Commit defc1823 authored 1 month ago by joeld

Downloads
- Patches
- Plain Diff

working pipline for parallel processing, added modular processing logic

parent 37fb8aad

No related branches found

No related tags found

No related merge requests found

Changes 4

Hide whitespace changes

Inline Side-by-side

Showing

data_loader.py 39 additions, 20 deletions

data_loader.py
load_data.py 7 additions, 5 deletions

load_data.py
main.py 99 additions, 0 deletions

main.py
shard_processor.py 52 additions, 0 deletions

shard_processor.py

with 197 additions and 25 deletions

Loading

0% Loading or .

You are about to add 0 people to the discussion. Proceed with caution.

Finish editing this message first!

Please register or sign in to comment