Remove Top Rows and Combine Data from Multiple Excel Files

Ғылым және технология

Check out our newly launched M Language course ↗️ - goodly.co.in/learn-m-powerquery/
Here is a solution to a common problem where you can remove a fixed or variable number of top junk rows from your data before you combine it from multiple excel files or datasets.
- - - - My Courses - - - -
✔️ Mastering DAX in Power BI -
goodly.co.in/learn-dax-powerbi/
✔️ Power Query Course-
goodly.co.in/learn-power-query/
✔️ Master Excel Step by Step-
goodly.co.in/learn-excel/
✔️ Business Intelligence Dashboards-
goodly.co.in/learn-excel-dash...
- - - - Combine Multiple Datasets Videos - - - -
Unpivot and Combine - • Unpivot and Combine Da...
Dynamic Combine Multiple Excel Files - • Combine Data From Mult...
Dynamic Combine Multiple Excel Sheets - • Combine Data from Mult...
---
► Artist Attribution
Music By: "After The Fall"
Track Name: "Tears Of Gaia"
Published by: Chill Out Records
- Source: goo.gl/fh3rEJ​
Official After The Fall KZread Channel Below
kzread.info/dron/GQE.html...
License: Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0)
Full license here: creativecommons.org/licenses

Пікірлер: 113

  • @GoodlyChandeep
    @GoodlyChandeep2 жыл бұрын

    Download Solution - www.goodly.co.in/wp-content/uploads/2021/09/Remove-Rows-and-Combine.zip

  • @wayneedmondson1065
    @wayneedmondson10652 жыл бұрын

    Awesome Chandeep! Your solutions are always creative and efficient. Thanks for sharing how to solve this problem. Thumbs up!! PS - Here is another method using Table.Skip() and List.PositionOf().. assuming that the column header label "Date" is not found above in the junk area of Column1 of any of the tables: = Table.AddColumn(Source, "Custom", each Table.PromoteHeaders(Table.Skip([Data],List.PositionOf([Data][Column1],"Date"))))

  • @KimHongTan

    @KimHongTan

    2 жыл бұрын

    Thanks, your tip just save my day! 🤗

  • @wayneedmondson1065

    @wayneedmondson1065

    2 жыл бұрын

    @@KimHongTan Awesome! Glad to help :))

  • @mathewinmuscat
    @mathewinmuscat2 жыл бұрын

    Chandeep, I don't know how to thank you....this has been a lifesaver for me. Thanks a ton.

  • @Nitin-yd5ko
    @Nitin-yd5ko Жыл бұрын

    Much informative ,,people like to hear in hindi too

  • @dirkstaszak4838
    @dirkstaszak48382 жыл бұрын

    Loved this one. Not only the described solution, but also the technique behind it. Thanks a lot

  • @DanteCry23
    @DanteCry23 Жыл бұрын

    Man this in incredible effective and well explained! I would ask you: What if I have junk between rows (Example: junk junk data data junk junk data data). I have not been able to figure it out. Thank you!

  • @rahulkalingeri1206
    @rahulkalingeri12062 жыл бұрын

    Amazing Thanks a lot.. I was modifying the input files manually to remove the junk rows This helps a lot 🙂

  • @ExcelWithChris
    @ExcelWithChris Жыл бұрын

    Thank you so much!! Greetings from South Africa. I love these short informative videos addressing specific every day issues one has with data.

  • @GoodlyChandeep

    @GoodlyChandeep

    Жыл бұрын

    Thanks!

  • @cristian.angyal
    @cristian.angyal2 жыл бұрын

    Nice trick! Thanks for sharing Chandeep!

  • @abeerattia4523
    @abeerattia45232 жыл бұрын

    Excellent , thanks for sharing your knowledge, Abeer from Egypt

  • @SamiElzaim
    @SamiElzaim2 жыл бұрын

    Now that is a clever and useful trick! Thanks chandeep!

  • @GrainneDuggan_Excel
    @GrainneDuggan_Excel2 жыл бұрын

    Interesting solution. Thanks Chandeep

  • @PrakashFatania
    @PrakashFatania Жыл бұрын

    Loved this video, quick and to the point - thanks

  • @GoodlyChandeep

    @GoodlyChandeep

    Жыл бұрын

    Glad you enjoyed it!

  • @seshakv
    @seshakv2 жыл бұрын

    Thank you. You are insanely genius. 👍👍👍👍

  • @ashokkumar-qy3ph
    @ashokkumar-qy3ph2 жыл бұрын

    Your Content is super simple

  • @gennarocimmino
    @gennarocimmino Жыл бұрын

    Very useful indeed. Impeccably explained. You are really good, brilliantly good.

  • @GoodlyChandeep

    @GoodlyChandeep

    Жыл бұрын

    Glad it was useful !

  • @MrAbrandao
    @MrAbrandao18 күн бұрын

    Fantástic work. But as ever , "we want more." Would be nice not to have to create a list with all the headers, but just one from a specific column. Also would be nice to remove empty columns..

  • @jpvramanarao1900
    @jpvramanarao1900 Жыл бұрын

    Thanku for valuable awasome tutorial and how to remove nth rows for multiple excel files

  • @learningforever1195
    @learningforever11952 жыл бұрын

    Thanks for another informative tutorial. This saves a lot of steps and simplifies it. An ask - In case of the excel file(s) with sub-totals in it, its been observed that the header record is not available and needs to be addressed manually. Is there any way around for same without manually removing the sub-totals from each excel file before importing in Power query. Thanks.

  • @vineetyoutubr5
    @vineetyoutubr52 жыл бұрын

    Thank you Chandeep 😊

  • @moonrollingpapers6521
    @moonrollingpapers65212 жыл бұрын

    HI, Thanks for the video, what about mutliable workbook instead of sheet in one workbook. when I do it it show "record" instead of " table" which we need to work on it

  • @EricaDyson
    @EricaDyson Жыл бұрын

    Great, as always and so very very useful! Thanks a lot.

  • @GoodlyChandeep

    @GoodlyChandeep

    Жыл бұрын

    Glad it was helpful!

  • @epernayjfl
    @epernayjfl2 жыл бұрын

    Mille mercis pour ce partage (et pour le fichier !) 👍

  • @GoodlyChandeep

    @GoodlyChandeep

    2 жыл бұрын

    De rien

  • @santoshpv321
    @santoshpv321 Жыл бұрын

    This is next level...Top Gun!!!

  • @GoodlyChandeep

    @GoodlyChandeep

    Жыл бұрын

    Thanks!

  • @mnowako
    @mnowako2 жыл бұрын

    Excellent! 👍

  • @sircorn4248
    @sircorn42482 жыл бұрын

    Simple makes awesome. Thanks a lot

  • @GoodlyChandeep

    @GoodlyChandeep

    2 жыл бұрын

    Glad you like it!

  • @kirshnannagarajan8756
    @kirshnannagarajan87563 ай бұрын

    It’s useful. Can you show how to pick files from share point with junk row and dynamic header. So I need to call out all the header name

  • @abdullah_alhathloul_
    @abdullah_alhathloul_2 жыл бұрын

    Thanks a lot..

  • @pranoysingha3236
    @pranoysingha32369 ай бұрын

    Thanks a lot

  • @DavidWarbeck
    @DavidWarbeck2 жыл бұрын

    Thank you so much for this - it is a genius solution!

  • @GoodlyChandeep

    @GoodlyChandeep

    2 жыл бұрын

    Glad you like it 💚

  • @BestICan
    @BestICan2 жыл бұрын

    Outstanding!

  • @brianxyz
    @brianxyz7 ай бұрын

    Amazing!

  • @jjohn108
    @jjohn1082 жыл бұрын

    Wow..!! This is very useful :)

  • @anshikapandey2977
    @anshikapandey29772 жыл бұрын

    Thanks for this informative video!! I would like to ask if you can make a video on conditional formatting chart as well.:D

  • @KgasS
    @KgasS Жыл бұрын

    👌. Normally PQ takes care of the field orders while appending if the data set is clean. If the field order is changed for example in one sheet change the order of Category and Value, the combined data result is different. A check for contains may be required in that case. Thanks

  • @brij26579
    @brij265792 жыл бұрын

    Awesome 👍

  • @eslamfahmy87
    @eslamfahmy87 Жыл бұрын

    Thank you for another wonderful tutorial, but it shall be write all column headers we have or only some of them are enough?

  • @dianaadams5026
    @dianaadams5026 Жыл бұрын

    Thanks!

  • @GoodlyChandeep

    @GoodlyChandeep

    Жыл бұрын

    Thanks Diana :)

  • @pdadey
    @pdadey2 жыл бұрын

    Thanks Chandeep for another insightful video. How would you provide a list of quoted column names ie "Date","Category", "Value" in a Custom Column?

  • @rajanmehandiratta596
    @rajanmehandiratta59611 ай бұрын

    Hi Chandeep, Thank you for this wonderful video. What if number of columns are more, then it's quite tedious to put all the column names manually in the list. Is there any way to link it with one list having detail of all the column headers to make it dynamic.

  • @vidyadharvm4132
    @vidyadharvm4132 Жыл бұрын

    Super thanks

  • @GoodlyChandeep

    @GoodlyChandeep

    Жыл бұрын

    Welcome Vidyadhar!

  • @rrrraaaacccc80
    @rrrraaaacccc80 Жыл бұрын

    Excelent 💯👍

  • @GoodlyChandeep

    @GoodlyChandeep

    Жыл бұрын

    Thanks!

  • @shoaibrehman9988
    @shoaibrehman99882 жыл бұрын

    Superb

  • @bravucod
    @bravucod2 жыл бұрын

    Brilliant

  • @manasa5639
    @manasa56392 жыл бұрын

    Nice and helpful

  • @GoodlyChandeep

    @GoodlyChandeep

    2 жыл бұрын

    Glad it was helpful 😊

  • @manasa5639

    @manasa5639

    2 жыл бұрын

    @@GoodlyChandeep Am becoming quite a fan of you/ your contents 😊👍🏼

  • @seanw4687
    @seanw46872 жыл бұрын

    Awsum Chandeep! You make it look so simple ;). A question though, how will merged header cells affect the query application? Thank you for the teaching.

  • @GoodlyChandeep

    @GoodlyChandeep

    2 жыл бұрын

    The merged header gets unmerged automatically in Power Query

  • @SamehRSameh
    @SamehRSameh2 жыл бұрын

    Magic ✨✨

  • @shenuforu
    @shenuforu5 ай бұрын

    Thanks a lot.. am currently workin on an assignment where this helps me however i hav one more instance where the heading comes bac again at the bottom wit few more values and currently my query ignores that.. is there a way i can apply the same principle multiple times so it picks tht data as well.. pls help..

  • @karanbirsoin9983
    @karanbirsoin9983 Жыл бұрын

    Hi Chandeep, will implement your method. My data has some blank lines also. I think your method reads all the records. Should work…..

  • @karanbirsoin9983

    @karanbirsoin9983

    Жыл бұрын

    Finally did it for cleaning the bank statement. One takeaway is that headers in data must match that are built into the custom column...

  • @smartwork4768
    @smartwork47682 жыл бұрын

    Hello Genius, I never miss your any of the videos and you are absolutely amazing in this. Well, i have one query if you could guide me here, - Let's say i have load query from folder and i have multiple files available in that folder belongs to every month sales and let's say i have built one report and loaded in excel using power query. - Now after deleting a few files from folder, how could i keep old file data in excel where i have loaded the query besides that file get removed from folder? - as we know after refracing of query it would load fresh data only in model. basically, i want to preserve old data also in my excel where i have load the query. Thanking you in advance.

  • @aahmed7749
    @aahmed7749 Жыл бұрын

    I am importing data from multiple excels in a folder. Each of them has two sheets, I just need first one. I was able to import data but when I use your formula to remove top 4 rows, it didn't work. I tried to do it before loading data (by going directly to EDIT option) and still didn't work.

  • @SuperUddie
    @SuperUddie8 ай бұрын

    Hello Chandeep! Thank you so much for the detailed explanation in this video and all the other videos which you record. I was able to unblock myself in multiple occasions following through your videos. Could you please help me on the below problem I am facing. I have a scenario where I need to combine 12 .xlsb (binary files) and each of these files have more than 300 columns. When I load these files into Power BI and try to combine, I am able to get only 255 columns. Is there any way where I can combine these files and get more than 255 columns (300 columns) Thank you again!

  • @The_Code_Father
    @The_Code_Father2 жыл бұрын

    Very smart mr chandeep but i was wondering what if columns header titles are dynamic and I make them data driven not manually insert them into the formula

  • @dorotkak.7598
    @dorotkak.75989 ай бұрын

    What if I have to also analyse several information from the rows where there is identification information regarding group columns below.

  • @polemack
    @polemack Жыл бұрын

    Great video ! But how to deal with nth junk rows at the end of the valid data ? Best wishes from Brazil. Thank you !

  • @icyknightster12
    @icyknightster12 Жыл бұрын

    HI Your video and explanation is very details. But I have issue that cant extract the table data on my 3 workbooks. I have header started from 8th row and I tried to replace the " each Record.ToList(_) {'Date', 'Category', 'Value'} " from Date to let say "Device owner" etc to my own but it couldnt show any table info. Any advise?

  • @koteswaribapatla8075
    @koteswaribapatla80752 жыл бұрын

    Hi,if I have more no of columns it is difficult to enter manually so how to automate even header names comparision with junk columns

  • @d20207
    @d20207 Жыл бұрын

    This is su f*cking useful!!! Thanks a lot for the detailed explanation.

  • @GoodlyChandeep

    @GoodlyChandeep

    Жыл бұрын

    Glad it helped!

  • @cheikhhakim5864
    @cheikhhakim5864 Жыл бұрын

    please Chandeep how can we do if the table to combine have differents headers how can we match them in M language before to combine

  • @vijaykumarjain3222
    @vijaykumarjain32227 ай бұрын

    Nice video is their any video in Hindi for better understand

  • @GoodlyChandeep

    @GoodlyChandeep

    7 ай бұрын

    www.youtube.com/@desigoodly/videos

  • @rlds07
    @rlds07 Жыл бұрын

    Is it possible for me to select columns if the condition of a row is met?

  • @rencerivera9171
    @rencerivera9171 Жыл бұрын

    thumbs up!

  • @GoodlyChandeep

    @GoodlyChandeep

    Жыл бұрын

    Thanks Rence!

  • @RaviGupta-mo1nf
    @RaviGupta-mo1nf2 жыл бұрын

    Lets add another variable to the problem. There is a 4th or 5th column in one or more sheets. These additional columns need not be named same across sheets. I want to ONLY combine only 3 columns which I select either manually or by making a list. How do I do that?

  • @md.raselhossain5086

    @md.raselhossain5086

    2 жыл бұрын

    The same question of me.

  • @dyanzzz
    @dyanzzz11 ай бұрын

    Do you know how to remove specific rows like for example i have 25 rows and i want to delete rows 6,9, 12-18... how to do that?

  • @priyankabhutada6198
    @priyankabhutada6198 Жыл бұрын

    I am getting my table empty with just headers. Applied same steps as you said @chandeep. What could be the reason.. Plz help

  • @pk5134
    @pk51342 жыл бұрын

    Thanks Chandeep. This was very useful! In this example the column headings were the same. How would you use this technique when the inner tables are different, say for example, the tables are Sales, order, product and, territory, where the column names are not the same. One way could be searching for a blank row before the main data begins. But this wouldn't work when there are no blank rows between the junk rows and the main data table Also, one concept I find hard to grasp is the use of _ and where it can be used, and how it differs from the "each" keyword. Could you please do a video on it ? Thanks!

  • @GoodlyChandeep

    @GoodlyChandeep

    2 жыл бұрын

    If the headers are different, I would then look for a completely filled record (which will be the header row). I'll post a video on each and _ soon. Thanks for the suggestion!

  • @pk5134

    @pk5134

    2 жыл бұрын

    @@GoodlyChandeep thank you!

  • @rlds07

    @rlds07

    Жыл бұрын

    @@GoodlyChandeep

  • @rlds07

    @rlds07

    Жыл бұрын

    @@GoodlyChandeep This is my friend, this video when the headers are different, did you record it? Hugs!

  • @brij26579
    @brij265792 жыл бұрын

    What if i have more than 10 columns & want to keep a check only on 1st column = Date to identify 1st Row & delete all previous rows

  • @jianlinchen7978
    @jianlinchen797810 ай бұрын

    Table.Skip(source, (x)=> x[Column1]"Date")

  • @gabrielabraham5512
    @gabrielabraham5512 Жыл бұрын

    Hi Chandeep

  • @gabrielabraham5512

    @gabrielabraham5512

    Жыл бұрын

    Table.Skip({data]) , each Record.ToList(_) {"Date", "null","Category") ) its not Working please help

  • @vashisht1
    @vashisht12 жыл бұрын

    I combined the data from all the sheets and then filter out junk using using filter than loaded the data

  • @alidaas
    @alidaas Жыл бұрын

    Sorry, it didn't work for me. I am consolidating .csv files from a folder, and unable to get rid of top 2 rows

  • @eslamfahmy87

    @eslamfahmy87

    Жыл бұрын

    I think CSV files are different from xlsx related to usage of M- code function at PQ

  • @pradnyajadhav189
    @pradnyajadhav1892 жыл бұрын

    What to do if table contains 50 columns, thn i have to write the 50 columns name into m query for check?

  • @cristian.angyal

    @cristian.angyal

    2 жыл бұрын

    Regardless of the number of rows you could always have a separate table with the Names of the columns, import it to PQ, make it a list and use it INSEATD of the manually input list shown by Chandeep in the video

  • @RaviGupta-mo1nf

    @RaviGupta-mo1nf

    2 жыл бұрын

    @@cristian.angyal Yes. This is what I did too. Make a List of Headers by having another query from any file. Is there a better way ? Also...What if the data does not start from Column A ? How do we deal with this ?

  • @iamdonna5918
    @iamdonna59182 жыл бұрын

    Hi Goodly 🙂 I tried doing it for various csv files but the custom column reverts error 😔

  • @GoodlyChandeep

    @GoodlyChandeep

    2 жыл бұрын

    Excel.Workbook() won't work for CSV files. for CSV files - use the function Csv.Document([Content])

  • @iamdonna5918

    @iamdonna5918

    2 жыл бұрын

    😃 Thank you for your reply! really appreciate it 🥰 your channel is very helpful!

  • @Rkeev1
    @Rkeev12 жыл бұрын

    Why not use index column drill down to the “value” header then remove rows and replace hard coded number with the drill down value

  • @dirkstaszak4838

    @dirkstaszak4838

    2 жыл бұрын

    As they say many ways lead to Rome. At the end of the day the choice would be on the more performant solution. There is no right or wrong

  • @krishnamanjunatha6183
    @krishnamanjunatha61832 жыл бұрын

    for more than 3 column how to hard code that in Table.ToRecord(_) {"col1","col2","col3"......}

  • @GoodlyChandeep

    @GoodlyChandeep

    2 жыл бұрын

    Use a list to define your columns!

  • @LaloinLondon
    @LaloinLondon Жыл бұрын

    What if you have a file with 70 columns? You type an array of 70 words?

  • @GoodlyChandeep

    @GoodlyChandeep

    Жыл бұрын

    It is pretty easy to copy the columns and surround them with curly braces and quotes. Then stick them in the query.

  • @LaloinLondon

    @LaloinLondon

    Жыл бұрын

    @@GoodlyChandeep Many thanks. It makes sense. I find your videos very helpful.

  • @Simon-vc1wk
    @Simon-vc1wk Жыл бұрын

    I have a similar scenario. The range of files I have (I dont have control over) will contain duplicate months in different workboooks, some files have single month other have month to date. I only want to import a single instance of a given month. My exception however is the meta data for my various files isn't unique other I'd use this to import my files. My thoughts is to count occurance of each list and exclude >2 or create a key (combination of all fields in ach file). Are there other ways to exclude duplicate months?

  • @chandeepchhabra
    @chandeepchhabra2 жыл бұрын

    Download Solution - www.goodly.co.in/wp-content/uploads/2021/09/Remove-Rows-and-Combine.zip

Келесі