Identify Duplicate Values in a Table with Power Query

This video covers how to identify duplicates with Power Query. Usually finding duplicates can be done with other Excel features like Conditional Formatting, but if you had a large data set (like hundreds of thousands of rows), you may find Excel slow. Power Query can handle large data sets so it's perfect for this scenario.
🔔 SUBSCRIBE to my channel ➜ goo.gl/wN3c3p
🏫 Excel Training ➜ www.exceltraining101.com/p/tr...
📚 Excel Books & Tech Gear ➜ www.amazon.com/shop/dough
⚙️ Tools: Screencasting ➜ techsmith.z6rjha.net/5Qe53
⚙️ Tools: Microsoft Office ➜ microsoft.msafflnk.net/rKL0G
⚙️ Tools: TubeBuddy ➜ www.tubebuddy.com/et101
📝 This description may contain affiliate links and we'll receive a small commission if a purchased is made using the links (but at no additional cost to you). It'll support the channel and so more videos like this can be made. Thanks for your support!
🎁 If you find these videos useful and want to support my channel go to www.buymeacoffee.com/dough
#excel
#msexcel
#doughexcel
~-~~-~~~-~~-~
Please watch: "Convert Table in a PDF File to Excel"
• Convert Table in a PDF...
~-~~-~~~-~~-~

Пікірлер: 45

  • @michaelrodriguez5452
    @michaelrodriguez54523 жыл бұрын

    Nice video sir. Great job explaining and giving the extra tips along the way !

  • @DougHExcel

    @DougHExcel

    3 жыл бұрын

    Glad you liked it

  • @lawrenceliew5812
    @lawrenceliew58122 жыл бұрын

    Thanks, Doug that's what I was looking for!

  • @DougHExcel

    @DougHExcel

    2 жыл бұрын

    Glad I could help!

  • @slocum7039
    @slocum70394 ай бұрын

    This was amazing Doug!! Subscribed immediately! Problem solved!

  • @DougHExcel

    @DougHExcel

    4 ай бұрын

    Thanks for the sub!

  • @BestICan
    @BestICan6 жыл бұрын

    Great tip, Doug.

  • @DougHExcel

    @DougHExcel

    6 жыл бұрын

    Hi Randy Smith, thanks for the comment!

  • @ElWiwif
    @ElWiwif2 жыл бұрын

    Thanks buddy! Your video still rocks!!

  • @DougHExcel

    @DougHExcel

    2 жыл бұрын

    Thanks! You rock!

  • @qtsog
    @qtsog2 жыл бұрын

    Thank you so much for your tutorial!

  • @DougHExcel

    @DougHExcel

    2 жыл бұрын

    You’re welcome!

  • @FRANKWHITE1996
    @FRANKWHITE19962 жыл бұрын

    Thats exactly what i needed today! Thanks

  • @DougHExcel

    @DougHExcel

    2 жыл бұрын

    You’re welcome!!

  • @FRANKWHITE1996

    @FRANKWHITE1996

    2 жыл бұрын

    @@DougHExcel Thanks again! 🙌🔥🙏

  • @stevennye5075
    @stevennye50754 жыл бұрын

    Thanks!

  • @DougHExcel

    @DougHExcel

    4 жыл бұрын

    You’re welcome!

  • @tapiwanalwimba6065
    @tapiwanalwimba60655 ай бұрын

    Thank you❤

  • @DougHExcel

    @DougHExcel

    5 ай бұрын

    You're welcome 😊

  • @arjunmv2830
    @arjunmv28307 ай бұрын

    Thank you sir

  • @DougHExcel

    @DougHExcel

    7 ай бұрын

    Welcome!

  • @joshuafancher3111
    @joshuafancher31115 жыл бұрын

    Thanks

  • @DougHExcel

    @DougHExcel

    5 жыл бұрын

    You’re welcome!

  • @Sket2Cr
    @Sket2Cr7 жыл бұрын

    Hi Dough! If I wanted to flag the duplicates in Power Query. Say a calculated column, the first line would have a 1 and all duplicates below it should show a 0. Is this possible in PQ?

  • @DougHExcel

    @DougHExcel

    7 жыл бұрын

    This vid might give some insights kzread.info/dash/bejne/gmWD3K2Qh7rWhrw.html

  • @olaayorinde6865
    @olaayorinde68657 жыл бұрын

    Hi DougThanks for this video. How can I not only identify duplicate but remove them using power query. I was trying to use COUNTIF function to label first occurrence as 0 and other will follow as 1,2,3...Is there a power query formula to the duplicate in number of occurrence the way countif fn will count them in excel sheet. Thanks

  • @olaayorinde6865

    @olaayorinde6865

    7 жыл бұрын

    Thank you Doug, pls can you paste this link again because there were no video in the above link you pasted.

  • @DougHExcel

    @DougHExcel

    7 жыл бұрын

    Try this.. kzread.info/dash/bejne/gmWD3K2Qh7rWhrw.html

  • @olaayorinde6865

    @olaayorinde6865

    7 жыл бұрын

    Thanks

  • @shoppersdream
    @shoppersdream3 жыл бұрын

    This was really nice. Can I split this workbook into 2 worksheets with PowerQuery? One with Duplicates and the other No Duplicates? Thanks

  • @DougHExcel

    @DougHExcel

    3 жыл бұрын

    Yes you can. Just reference the first one you created and make that the duplicate or non duplicate query (diff name) and load to sheet.

  • @shoppersdream

    @shoppersdream

    3 жыл бұрын

    @@DougHExcel Thank You!

  • @markhealy4156
    @markhealy41565 жыл бұрын

    Can it be done for multiple columns, I'd be working with sales files where one column will always have duplicates such as invoice numbers. I wouldn't be removing those. I would like to remove a row of 20 columns where the entire row is an obvious duplicate that must be removed.

  • @DougHExcel

    @DougHExcel

    3 жыл бұрын

    Yep, it can

  • @Ingles4allYou
    @Ingles4allYou5 жыл бұрын

    How to compare two big excel files by power query? Sheet1.xls and Sheet2.xls Both haveThousand rows and more than 50 columns each . they have complete different columns unless 2 coluns C end D are common in both. .I want to have sheet 3 with all lines and coluns of sheet1 except all lines( when the line C and D lines) matchs sheet 2 ? Pls. Basically i want extract the difference Thanks

  • @DougHExcel

    @DougHExcel

    5 жыл бұрын

    Maybe this one will give some insight kzread.info/dash/bejne/eaCBqKSHgbi3gZs.html

  • @wadieseruge3523
    @wadieseruge35238 жыл бұрын

    hi Doug, if I want cell a2 change it's cell color, every time cell a1, shows the word N/A. what formula would I need to use ? thank you. dan

  • @DougHExcel

    @DougHExcel

    8 жыл бұрын

    I'm Assuming you are using the conditional formatting feature (kzread.info/dash/bejne/n5mjydF6j7C-nco.html). You could use the errors to define the conditional format if you meant #N/A. That is an error condition that you can define a conditional format. If it is just N/A, use the condition criteria in quotes (i.e., text equals "N/A")

  • @wadieseruge3523

    @wadieseruge3523

    8 жыл бұрын

    Doug, Hi. Im not sure if i explained my self correctly. Here is what im trying to do, is the following: When cell A-1 shows the word "split-shift", i would like the cell below that, that is A-2, and A-3 to change its cell color to the word "open". Which part of the conditional format option/function would you recommend me to try, to make this work. Thank you. Thank you for the fast respond earlier. Dan

  • @arjunmv2830
    @arjunmv28307 ай бұрын

    How to do look up in power query for large data sir.. Data exceeded excel maximum limit

  • @DougHExcel

    @DougHExcel

    7 ай бұрын

    Try to use CSV file as your source. However the output tab will have the constraint.

  • @ManishKumar-qj7sq
    @ManishKumar-qj7sq3 жыл бұрын

    how to find partial duplicate in power query

  • @DougHExcel

    @DougHExcel

    3 жыл бұрын

    maybe one of these related can give insight kzread.info/dash/bejne/mK6asJptm6fXl7g.html

  • @ManishKumar-qj7sq

    @ManishKumar-qj7sq

    3 жыл бұрын

    @@DougHExcelthanks..bt this is not what I want..

  • @JJ_TheGreat
    @JJ_TheGreat2 жыл бұрын

    Why not just go to "Keep Rows" > "Keep Duplicates" - to identify them... Easy peasy. Then you can always X out the step it creates after identifying them.

Келесі