How can we help you today? How can we help you today?

Mapping SQL tables and Shuffle data ?

I am using an external data base view to map. This works fine where the records are inserted in the same order as whats on the source db.

Is their a Shuffle option. I need to import 2 columns and each has to match. For example let assume I am trying to do a movie ticket sales table.

Movie Name, Studio Name (so both has to match) are mapped from external source. Right now it works where col1 and and col2 mapes the source table..but in the same order. Their is no shuffle and random way to do it

Jay
jay23
0

Comments

5 comments

  • benhall
    Hello,

    I recommend you to use the SQL Statement generator as this will shuffle the data for you. You should be able to mix and match the SQL Statement Generator with the Map functionality, allowing you to only shuffle the columns you are interested in.

    I hope this helps.

    Ben
    benhall
    0
  • jay23
    benhall wrote:
    Hello,

    I recommend you to use the SQL Statement generator as this will shuffle the data for you. You should be able to mix and match the SQL Statement Generator with the Map functionality, allowing you to only shuffle the columns you are interested in.

    I hope this helps.

    Ben

    Ben,

    Right now on both columns i am populating values usng SQL Statment Generator...how ever I cant find the MAP function .

    It looks like we can refer to the table which we are trying to populate on a WHERE clause. So in what order is the data is generated. If I want 1000 records....do you go row by row or first fill the first column and then go to the second column

    jay
    jay23
    0
  • benhall
    With the SQL Statement generator, by default we shuffle the data, by unticking shuffle data the data will be inserted in the same order it appears in the database.

    Hope this helps.

    Ben
    benhall
    0
  • jay23
    benhall wrote:
    With the SQL Statement generator, by default we shuffle the data, by unticking shuffle data the data will be inserted in the same order it appears in the database.

    Hope this helps.

    Ben

    Ben

    Here is what I am trying to do.

    Lets say the first column on the generated table is First Name and this is read from another table. The second field on the generated table is last name, for this I need to make sure the data I generate match the first column.

    I tried to do some thing like this but it did not work

    Select OrginalTable.SecondCol
    From OrginalTable,GeneratedTable
    where GeneratedTable.FirstCol = OrginalTable.First Col.
    jay23
    0
  • Dan Baron

    It looks like your mapping relies on the order of inserts matching the source DB, which can be tricky when dealing with shuffle operations. If the data gets shuffled or processed in parallel, maintaining the expected sequence might break. A good approach is to use a unique identifier or timestamp for ordering instead of relying on the default insert sequence.

    Speaking of mapping, this reminds me of how apps like Castle TV App organize and stream content. Just like SQL tables need proper indexing for efficient retrieval, streaming apps map content dynamically to ensure smooth playback without delays. If data isn’t structured properly, both in databases and streaming platforms, things can get messy fast!

    Dan Baron
    0

Add comment

Please sign in to leave a comment.