4

This script is giving me an error because it consumes too much resources. What can I do to fix that?

Dim oSht As Worksheet
Dim i As Long, j As Integer
Dim LRow As Long, LCol As Long
Dim Email1Col As Integer, Email2Col As Integer, Email3Col As Integer
Dim arr As Variant
Dim SplEmail3 As String


'Definitions
Set oSht = ActiveSheet
Email1Col = 6
Email2Col = 7
Email3Col = 8
'-----------

With oSht
'LRow = .Range("G" & .Rows.Count).End(xlUp).Row
LRow = 1048576
'LCol = .Cells(1, .Columns.Count).End(xlToLeft).Column
End With

For i = 2 To LRow
    'If oSht.Rows(i + 1).EntireRow = 0 Then GoTo Skip
    If Cells(i, Email1Col).Value <> "" Or Cells(i, Email3Col).Value <> "" Then
        If Cells(i, Email2Col) <> "" Then
            'email2 to new row + copy other data
            Rows(i + 1).EntireRow.Insert
            oSht.Rows(i + 1).EntireRow.Value = oSht.Rows(i).EntireRow.Value
            Range(Cells(i + 1, Email1Col), Cells(i + 1, Email3Col)).ClearContents
            Cells(i + 1, Email1Col) = Cells(i, Email2Col)
            'email3 to new row + copy other data
        End If
        If Cells(i, Email3Col) <> "" Then
            arr = Split(Cells(i, Email3Col), ",", , 1)
            For j = 0 To UBound(arr)
                'split into single emails
                SplEmail3 = Replace((arr(j)), " ", "", 1, , 1)
                'repeat the process for every split
                Rows(i + 2 + j).EntireRow.Insert
                oSht.Rows(i + 2 + j).EntireRow.Value = oSht.Rows(i).EntireRow.Value
                Range(Cells(i + 2 + j, Email1Col), Cells(i + 2 + j, Email3Col)).ClearContents
                Cells(i + 2 + j, Email1Col) = SplEmail3
            Next j
        End If
        Range(Cells(i, Email2Col), Cells(i, Email3Col)).ClearContents
    Else
        Rows(i).EntireRow.Delete
    End If
Skip:
Next i

sample data:

col1, col2,..., col6, col7 ,  col8
name, bla, ...,mail1,mail2,(mail3,mail4,mail5)

needs to become this:

col1, col2,..., col6
name, bla, ...,mail1
TCN
  • 1,571
  • 1
  • 26
  • 46
  • 5
    `LRow = 1048576` Why would you do that? What exacty are you trying to achieve? – Siddharth Rout Jul 30 '16 at 13:51
  • sorry for the wait (still working on other functions for this spreadsheet...), I need it to normalize the email columns across all rows (which can potentially be more than 500.000) – TCN Jul 30 '16 at 14:23
  • 1
    Can you explain " normalize the email columns"? – Siddharth Rout Jul 30 '16 at 14:23
  • In other words: (1)move emails on col7 and col8 to col6 in a new line, making it a single column of emails (col6 containing all emails). (2) The rest of the data in the row should be copied to the new line. (3) some of the emails on col8 are on the same cell, separated by a comma, those should go each to it's own line too, like the others on col7. – TCN Jul 30 '16 at 14:26
  • Can you post a screenshot of the data. I am sure there is a much better and faster way to achieve it :) – Siddharth Rout Jul 30 '16 at 14:30
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/118700/discussion-between-jony-and-siddharth-rout). – TCN Jul 30 '16 at 14:34
  • 1
    @jony I would suggest trying `Power Pivot` or `Get and Transform` (which depends on your Excel Version) to just **UnPivot** the email columns. – Ron Rosenfeld Jul 30 '16 at 14:47
  • I have never used power pivot, I can work with Power Pivot or Get and Transform through vba right? – TCN Jul 30 '16 at 14:49
  • 1
    I don't know, I've not done that, nor automated the process of creating the query. Power Query (as it is generally called) is a different language, but it is built in to Excel since 2010, I believe. For your type of problem, it is trivial to create the query using the built in tools. – Ron Rosenfeld Jul 30 '16 at 14:52
  • "create the query using the built in tools" yes, but other people will need to use it over time, it needs to be something that I can call through vba :/ – TCN Jul 30 '16 at 14:56
  • 1
    Once you have created the query, it can be re used. I suggest you look at it. – Ron Rosenfeld Jul 30 '16 at 15:05
  • @SiddharthRout - [Is “Please post a picture” a non-constructive comment?](http://meta.stackoverflow.com/questions/316945/is-please-post-a-picture-a-non-constructive-comment). To the OP - [Why not images of Code and Sample Data](http://meta.stackoverflow.com/questions/285551/285557#285557) and [Discourage screenshots of code and/or errors](http://meta.stackoverflow.com/questions/303812). –  Jul 30 '16 at 15:18
  • Possible duplicate of [Combine rows with duplicate values in one cell and merge values in other cell](http://stackoverflow.com/questions/20227196/excel-vba-combine-rows-with-duplicate-values-in-one-cell-and-merge-values-in-o/32546352#32546352). –  Jul 30 '16 at 15:27
  • what I am trying to achieve is the exact opposite of that question (maybe that is exactly your point?). – TCN Jul 30 '16 at 15:33
  • 1
    Maybe this one: http://stackoverflow.com/a/10922351/293078. Also, remember to do like @DougGlancy when responding to comments so that the commenter is notified. – Doug Glancy Jul 30 '16 at 15:40
  • 1
    @DougGlancy that one seems helpful! – TCN Jul 30 '16 at 15:44
  • Will it solve my problem if I simply use arrays instead of calling the sheet every time? – TCN Jul 30 '16 at 15:58
  • @Jeeped I really don't care what that link says. If the user post an image of sample data in the question and which can help me to give him a better solution then I don't see a problem in that ;) – Siddharth Rout Jul 30 '16 at 16:03
  • @Siddharth Rout can you tell me if using arrays wherever possible will solve my problem? :/ – TCN Jul 30 '16 at 16:05
  • 1
    Yes using array is what I had in mind. In fact I am writing a code to test it and then will post an answer. – Siddharth Rout Jul 30 '16 at 16:06
  • 1
    @SiddharthRout - A lot of programmers have their own 'itchy sweater'; images of sample data is one of mine. I never see the sense of posting an image of sample data when everyone who views the problem has to retype the data to proof a solution. Besides, it is more effort to post an image then it is to paste some data from a worksheet into the question. –  Jul 30 '16 at 16:07
  • @Jeeped Well, I always test my code before posting it as I do not want to waste OP's and very specifically "my" time. I don't want to play ping pong with questions. I see the data and then simply post the solution... it has always worked for me ;) – Siddharth Rout Jul 30 '16 at 16:10
  • @Jeeped I also provided data typed in the question (the formatting is not the best, I have to look into that when I'm finished with this project). – TCN Jul 30 '16 at 16:11
  • 1
    Highlight (*aka Select*) the sample data and tap [ctrl]+K or make sure every line is indented by 4 spaces. –  Jul 30 '16 at 16:13
  • 1
    The opposite of that earlier link would be something like [Split delimited 2nd and 3rd column data into new rows](http://stackoverflow.com/questions/29856393/split-delimited-2nd-and-3rd-column-data-into-new-rows/29861673#29861673). The operative word to search for is **flatten**. –  Jul 30 '16 at 16:22
  • 1
    @Jeeped, I think that "normalize" and "unpivot", both used in this discussion, are equally operative, if not more so. – Doug Glancy Jul 30 '16 at 16:58
  • @siddarthrout, jeeped, Ron Rosenfeld, Doug Glancy: Thanks to you all, I was able to finish my project in a suitable timeframe (missed the deadline, though, but not by much). I now understand most of the concepts you showed me and the ones I don't yet fully understand I'll be sure to check them out asap (i.e. power query). By the way, your pieces of code and mine combined took 10 seconds for a sample of 50.000 lines. Thank you! (and please accept my apologies for all the trouble.) – TCN Jul 31 '16 at 20:14

2 Answers2

7

Note: I have tested this with very small piece of data.. Give it a try and if you are stuck then let me know. We will take it from there.

Let's say our data looks like this

enter image description here

Now we run this code

Sub Sample()
    Dim oSht As Worksheet
    Dim arr As Variant, FinalArr() As String
    Dim i As Long, j As Long, k As Long, LRow As Long

    Set oSht = ActiveSheet

    With oSht
        LRow = .Range("A" & .Rows.Count).End(xlUp).Row

        arr = .Range("A2:H" & LRow).Value

        i = Application.WorksheetFunction.CountA(.Range("G:H"))

        '~~> Defining the final output array
        ReDim Preserve FinalArr(1 To (LRow + i - 3), 1 To 6)

        k = 0
        For i = LBound(arr) To UBound(arr)
            k = k + 1
            FinalArr(k, 1) = arr(i, 1)
            FinalArr(k, 2) = arr(i, 2)
            FinalArr(k, 3) = arr(i, 3)
            FinalArr(k, 4) = arr(i, 4)
            FinalArr(k, 5) = arr(i, 5)
            If arr(i, 6) <> "" Then FinalArr(k, 6) = arr(i, 6)

            For j = 7 To 8
                If arr(i, j) <> "" Then
                    k = k + 1
                    FinalArr(k, 1) = arr(i, 1)
                    FinalArr(k, 2) = arr(i, 2)
                    FinalArr(k, 3) = arr(i, 3)
                    FinalArr(k, 4) = arr(i, 4)
                    FinalArr(k, 5) = arr(i, 5)
                    FinalArr(k, 6) = arr(i, j)
                End If
            Next j
        Next i

        .Rows("2:" & .Rows.Count).Clear

        .Range("A2").Resize(UBound(FinalArr), 6).Value = FinalArr
    End With
End Sub

Output

enter image description here

Siddharth Rout
  • 147,039
  • 17
  • 206
  • 250
  • 2 working answers, but you were a minute faster! It's not that a minute is important when posting an answer, but you were a minute faster! XD Also you did it without using arrays! I'll study both of your solutions. Thank you so much! – TCN Jul 30 '16 at 16:54
  • I did use arrays ;) `arr` and `FinalArr` are arrays – Siddharth Rout Jul 30 '16 at 16:58
  • I meant using arrays. – TCN Jul 30 '16 at 17:31
5

You can use Power Query. Your comment led me to do some testing, and that can be done while recording a macro. For example, assuming your data is in a "table":

Sub createPQ()

    ActiveWorkbook.Queries.Add Name:="Table1", Formula:= _
        "let" & Chr(13) & "" & Chr(10) & "    Source = Excel.CurrentWorkbook(){[Name=""Table1""]}[Content]," & Chr(13) & "" & Chr(10) & "    #""Changed Type"" = Table.TransformColumnTypes(Source,{{""FirstName"", type text}, {""LastName"", type text}, {""blah1"", type text}, {""b lah2"", type text}, {""bla3"", type text}, {""email1"", type text}, {""email2"", type text}, {""email3"", type text}})," & Chr(13) & "" & Chr(10) & "    #""Unpivoted Columns"" = Tab" & _
        "le.UnpivotOtherColumns(#""Changed Type"", {""FirstName"", ""LastName"", ""blah1"", ""b lah2"", ""bla3""}, ""Attribute"", ""Value"")" & Chr(13) & "" & Chr(10) & "in" & Chr(13) & "" & Chr(10) & "    #""Unpivoted Columns"""
    Sheets.Add After:=ActiveSheet
    With ActiveSheet.ListObjects.Add(SourceType:=0, Source:= _
        "OLEDB;Provider=Microsoft.Mashup.OleDb.1;Data Source=$Workbook$;Location=Table1" _
        , Destination:=Range("$A$1")).QueryTable
        .CommandType = xlCmdSql
        .CommandText = Array("SELECT * FROM [Table1]")
        .RowNumbers = False
        .FillAdjacentFormulas = False
        .PreserveFormatting = True
        .RefreshOnFileOpen = False
        .BackgroundQuery = True
        .RefreshStyle = xlInsertDeleteCells
        .SavePassword = False
        .SaveData = True
        .AdjustColumnWidth = True
        .RefreshPeriod = 0
        .PreserveColumnInfo = False
        .ListObject.DisplayName = "Table1_2"
        .Refresh BackgroundQuery:=False
    End With
End Sub

If your user adds data, and needs to refresh the query, Data RibbonConnection tabRefresh (or you could create a button to do that if you prefer).

The unknown is how it will work on a DB of your size.

-- Before

enter image description here

-- After

enter image description here

Ron Rosenfeld
  • 53,870
  • 7
  • 28
  • 60
  • Now I have 2 beautiful answers posted almost at the same time! I'll have to give it up to @Siddharth Rout because he was about a minute faster. But I love your solution too! Thank you! I'll be sure to study it and learn from you! – TCN Jul 30 '16 at 16:52
  • @jony See which one works better on your database. Both were tested with much less data than you are using. – Ron Rosenfeld Jul 30 '16 at 16:56
  • ++ I agree with ron here @jony. A minute faster doesn't mean a thing :D Test it with your complete database and then choose the best solution :) – Siddharth Rout Jul 30 '16 at 16:56
  • @RonRosenfeld ok, I'll increase my sample size and see if one performs better than the other. – TCN Jul 30 '16 at 16:57
  • @jony Please let us know if you have determined a speed difference. – Ron Rosenfeld Aug 01 '16 at 11:46
  • @RonRosenfeld ok, I've been so busy I haven't had the time to test things. I'll do it as soon as possible and then post the results. – TCN Aug 02 '16 at 12:37