How You.com Applies Spark + Hugging Face Transformers for Search at Scale

Ғылым және технология

A multi-group, Sept 13 2022 NLP meetup presentation, "How You.com Applies Spark + Hugging Face Transformers for Search at Scale," with Sam Bean and Saahil Jain of You.com, moderated by Seth Grimes.
For the New York Natural Language Processing meetup, visit www.meetup.com/ny-nlp/.

Пікірлер: 4

  • @VaclavKosar
    @VaclavKosar Жыл бұрын

    Great meetup, thx. Loved the part about Spark cpu transformer embedding calculation for semantic indexing.

  • @eugeneware3296
    @eugeneware3296 Жыл бұрын

    This is so brilliant. It's so hard to find these real-life learnings of how to deliver world-class ML systems in production. Especially the interaction between spark and transformers.

  • @jasonmackay891

    @jasonmackay891

    Жыл бұрын

    100%

  • @cosmicray007
    @cosmicray007 Жыл бұрын

    Question, why not just use Apache Hive as a Catalog meta database, like the one found in AWS Glue. No need to move the data, since the catalog mdb points to the data. Moving data is costly, just saying. 18:44 sounds like we are in a pickle :)

Келесі