Removing dupes in list of lists in Python










9















Basically, I'm trying to do remove any lists that begin with the same value. For example, two of the below begin with the number 1:



a = [[1,2],[1,0],[2,4],[3,5]]


Because the value 1 exists at the start of two of the lists -- I need to remove both so that the new list becomes:



b = [[2,4],[3,5]]


How can I do this?



I've tried the below, but the output is: [[1, 2], [2, 4], [3, 5]]



def unique_by_first_n(n, coll):
seen = set()
for item in coll:
compare = tuple(item[:n])
print compare # Keep only the first `n` elements in the set
if compare not in seen:
seen.add(compare)
yield item

a = [[1,2],[1,0],[2,4],[3,5]]

filtered_list = list(unique_by_first_n(1, a))









share|improve this question



















  • 2





    2 also exists in both. Why don't you remove them?

    – kharandziuk
    Aug 26 '18 at 19:20











  • good point -- i only need to remove where the first item is the same. so just the 1 in this case

    – Jamie Lunn
    Aug 26 '18 at 19:22















9















Basically, I'm trying to do remove any lists that begin with the same value. For example, two of the below begin with the number 1:



a = [[1,2],[1,0],[2,4],[3,5]]


Because the value 1 exists at the start of two of the lists -- I need to remove both so that the new list becomes:



b = [[2,4],[3,5]]


How can I do this?



I've tried the below, but the output is: [[1, 2], [2, 4], [3, 5]]



def unique_by_first_n(n, coll):
seen = set()
for item in coll:
compare = tuple(item[:n])
print compare # Keep only the first `n` elements in the set
if compare not in seen:
seen.add(compare)
yield item

a = [[1,2],[1,0],[2,4],[3,5]]

filtered_list = list(unique_by_first_n(1, a))









share|improve this question



















  • 2





    2 also exists in both. Why don't you remove them?

    – kharandziuk
    Aug 26 '18 at 19:20











  • good point -- i only need to remove where the first item is the same. so just the 1 in this case

    – Jamie Lunn
    Aug 26 '18 at 19:22













9












9








9








Basically, I'm trying to do remove any lists that begin with the same value. For example, two of the below begin with the number 1:



a = [[1,2],[1,0],[2,4],[3,5]]


Because the value 1 exists at the start of two of the lists -- I need to remove both so that the new list becomes:



b = [[2,4],[3,5]]


How can I do this?



I've tried the below, but the output is: [[1, 2], [2, 4], [3, 5]]



def unique_by_first_n(n, coll):
seen = set()
for item in coll:
compare = tuple(item[:n])
print compare # Keep only the first `n` elements in the set
if compare not in seen:
seen.add(compare)
yield item

a = [[1,2],[1,0],[2,4],[3,5]]

filtered_list = list(unique_by_first_n(1, a))









share|improve this question
















Basically, I'm trying to do remove any lists that begin with the same value. For example, two of the below begin with the number 1:



a = [[1,2],[1,0],[2,4],[3,5]]


Because the value 1 exists at the start of two of the lists -- I need to remove both so that the new list becomes:



b = [[2,4],[3,5]]


How can I do this?



I've tried the below, but the output is: [[1, 2], [2, 4], [3, 5]]



def unique_by_first_n(n, coll):
seen = set()
for item in coll:
compare = tuple(item[:n])
print compare # Keep only the first `n` elements in the set
if compare not in seen:
seen.add(compare)
yield item

a = [[1,2],[1,0],[2,4],[3,5]]

filtered_list = list(unique_by_first_n(1, a))






python python-2.7 list






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Aug 27 '18 at 2:16









Peter Mortensen

13.6k1985111




13.6k1985111










asked Aug 26 '18 at 19:18









Jamie LunnJamie Lunn

514




514







  • 2





    2 also exists in both. Why don't you remove them?

    – kharandziuk
    Aug 26 '18 at 19:20











  • good point -- i only need to remove where the first item is the same. so just the 1 in this case

    – Jamie Lunn
    Aug 26 '18 at 19:22












  • 2





    2 also exists in both. Why don't you remove them?

    – kharandziuk
    Aug 26 '18 at 19:20











  • good point -- i only need to remove where the first item is the same. so just the 1 in this case

    – Jamie Lunn
    Aug 26 '18 at 19:22







2




2





2 also exists in both. Why don't you remove them?

– kharandziuk
Aug 26 '18 at 19:20





2 also exists in both. Why don't you remove them?

– kharandziuk
Aug 26 '18 at 19:20













good point -- i only need to remove where the first item is the same. so just the 1 in this case

– Jamie Lunn
Aug 26 '18 at 19:22





good point -- i only need to remove where the first item is the same. so just the 1 in this case

– Jamie Lunn
Aug 26 '18 at 19:22












4 Answers
4






active

oldest

votes


















5














You can use collections.Counter with list comprehension to get sublists whose first item appears only once:



from collections import Counter
c = Counter(n for n, _ in a)
b = [[x, y] for x, y in a if c[x] == 1]





share|improve this answer

























  • that sounds overcomplicated

    – Jean-François Fabre
    Aug 26 '18 at 19:39











  • @Jean-FrançoisFabre its basically [[x,y] for key, counterValue in counterItems if counterValue == 1 for x, y in my_list if x == key]

    – Fabian N.
    Aug 26 '18 at 19:42







  • 1





    what annoys me is the double loop. The other answer has less O complexity (and also your rewrite with new variable names is clearer)

    – Jean-François Fabre
    Aug 26 '18 at 19:44






  • 1





    @Jean-FrançoisFabre Indeed thanks. I've fixed my answer accordingly, although it is now admittedly the same as Joe's answer.

    – blhsing
    Aug 26 '18 at 19:54











  • since it seems to be the best way, I don't see a problem. I just didn't want people copy an unefficient answer (even if it worked).

    – Jean-François Fabre
    Aug 26 '18 at 19:54


















6














An efficient solution would be to create a Counter object to hold the occurrences of the first elements, and then filter the sub-lists in the main list:



from collections import Counter
counts = Counter(l[0] for l in a)
filtered = [l for l in a if counts[l[0]] == 1]
#[[2, 4], [3, 5]]





share|improve this answer




















  • 2





    filtered = [l for l in a if counts[l[0]] == 1] is probably clearer but yes.

    – Jean-François Fabre
    Aug 26 '18 at 19:37











  • @Jean-FrançoisFabre Fair enough. Do you understand the logic in blhsing's answer? It certainly doesn't guarantee the order of the list is maintained.

    – Joe Iddon
    Aug 26 '18 at 19:38







  • 3





    no I don't but I see a loop which makes the solution overcomplex

    – Jean-François Fabre
    Aug 26 '18 at 19:38


















6














If you are happy to use a 3rd party library, you can use Pandas:



import pandas as pd

a = [[1,2],[1,0],[2,4],[3,5]]

df = pd.DataFrame(a)
b = df.drop_duplicates(subset=[0], keep=False).values.tolist()

print(b)

[[2, 4], [3, 5]]


The trick is the keep=False argument, described in the docs for pd.DataFrame.drop_duplicates.






share|improve this answer


















  • 3





    I never used pandas, always did things like that manually, like the other answers but THIS is another level of readability -> sparked my interest in the library -> have a +1

    – Fabian N.
    Aug 26 '18 at 19:48






  • 2





    @FabianN., I certainly don't recommend Pandas as a means of learning, but vectorised operations do have their use. And I would be disappointed to make people believe Python list + dict are the only way to structure data!

    – jpp
    Aug 26 '18 at 19:52












  • never used pandas myself, but I'm pretty sure it beats the pure python answers in terms of speed (well, don't count the import pandas part obviously :)

    – Jean-François Fabre
    Aug 26 '18 at 19:56











  • @FabianN. You should also checkout numpy - very fast, especially on large data sets like when image processing.

    – Joe Iddon
    Aug 26 '18 at 20:56












  • @JoeIddon I'm aware of numpy, I think that's the main reason I always ignored pandas. For simple things, I just used native python and for vectorized operations I used numpy so I never felt it was worth the effort to put even more layers on top... but from what I see above, with pandas, one can write code that's almost a sentence.

    – Fabian N.
    Aug 26 '18 at 21:27



















0














Solution 1



a = [[1,2],[1,0],[2,4],[3,5]]
b =
for item in a:
i = 0
if item[0] == a[i][0]:
i =+ 1
continue
else:
b.append(item)
i += 1


Solution 2



a = [[1,2],[1,0],[2,4],[3,5]]
b =

for item in a:
for i in range(0, len(a)):
if item[0] == a[i][0]:
break
else:
if item in b:
continue
else:
b.append(item)


Output




(xenial)vash@localhost:~/pcc/10$ python3 remove_help.py 
[[1, 2], [1, 0], [2, 4], [3, 5]]
[[2, 4], [3, 5]]



Achieved your goal no complex methods involed!
Enjoy!






share|improve this answer
























    Your Answer






    StackExchange.ifUsing("editor", function ()
    StackExchange.using("externalEditor", function ()
    StackExchange.using("snippets", function ()
    StackExchange.snippets.init();
    );
    );
    , "code-snippets");

    StackExchange.ready(function()
    var channelOptions =
    tags: "".split(" "),
    id: "1"
    ;
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function()
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled)
    StackExchange.using("snippets", function()
    createEditor();
    );

    else
    createEditor();

    );

    function createEditor()
    StackExchange.prepareEditor(
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader:
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    ,
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    );



    );













    draft saved

    draft discarded


















    StackExchange.ready(
    function ()
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f52029529%2fremoving-dupes-in-list-of-lists-in-python%23new-answer', 'question_page');

    );

    Post as a guest















    Required, but never shown

























    4 Answers
    4






    active

    oldest

    votes








    4 Answers
    4






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    5














    You can use collections.Counter with list comprehension to get sublists whose first item appears only once:



    from collections import Counter
    c = Counter(n for n, _ in a)
    b = [[x, y] for x, y in a if c[x] == 1]





    share|improve this answer

























    • that sounds overcomplicated

      – Jean-François Fabre
      Aug 26 '18 at 19:39











    • @Jean-FrançoisFabre its basically [[x,y] for key, counterValue in counterItems if counterValue == 1 for x, y in my_list if x == key]

      – Fabian N.
      Aug 26 '18 at 19:42







    • 1





      what annoys me is the double loop. The other answer has less O complexity (and also your rewrite with new variable names is clearer)

      – Jean-François Fabre
      Aug 26 '18 at 19:44






    • 1





      @Jean-FrançoisFabre Indeed thanks. I've fixed my answer accordingly, although it is now admittedly the same as Joe's answer.

      – blhsing
      Aug 26 '18 at 19:54











    • since it seems to be the best way, I don't see a problem. I just didn't want people copy an unefficient answer (even if it worked).

      – Jean-François Fabre
      Aug 26 '18 at 19:54















    5














    You can use collections.Counter with list comprehension to get sublists whose first item appears only once:



    from collections import Counter
    c = Counter(n for n, _ in a)
    b = [[x, y] for x, y in a if c[x] == 1]





    share|improve this answer

























    • that sounds overcomplicated

      – Jean-François Fabre
      Aug 26 '18 at 19:39











    • @Jean-FrançoisFabre its basically [[x,y] for key, counterValue in counterItems if counterValue == 1 for x, y in my_list if x == key]

      – Fabian N.
      Aug 26 '18 at 19:42







    • 1





      what annoys me is the double loop. The other answer has less O complexity (and also your rewrite with new variable names is clearer)

      – Jean-François Fabre
      Aug 26 '18 at 19:44






    • 1





      @Jean-FrançoisFabre Indeed thanks. I've fixed my answer accordingly, although it is now admittedly the same as Joe's answer.

      – blhsing
      Aug 26 '18 at 19:54











    • since it seems to be the best way, I don't see a problem. I just didn't want people copy an unefficient answer (even if it worked).

      – Jean-François Fabre
      Aug 26 '18 at 19:54













    5












    5








    5







    You can use collections.Counter with list comprehension to get sublists whose first item appears only once:



    from collections import Counter
    c = Counter(n for n, _ in a)
    b = [[x, y] for x, y in a if c[x] == 1]





    share|improve this answer















    You can use collections.Counter with list comprehension to get sublists whose first item appears only once:



    from collections import Counter
    c = Counter(n for n, _ in a)
    b = [[x, y] for x, y in a if c[x] == 1]






    share|improve this answer














    share|improve this answer



    share|improve this answer








    edited Aug 26 '18 at 19:52

























    answered Aug 26 '18 at 19:27









    blhsingblhsing

    31.5k41336




    31.5k41336












    • that sounds overcomplicated

      – Jean-François Fabre
      Aug 26 '18 at 19:39











    • @Jean-FrançoisFabre its basically [[x,y] for key, counterValue in counterItems if counterValue == 1 for x, y in my_list if x == key]

      – Fabian N.
      Aug 26 '18 at 19:42







    • 1





      what annoys me is the double loop. The other answer has less O complexity (and also your rewrite with new variable names is clearer)

      – Jean-François Fabre
      Aug 26 '18 at 19:44






    • 1





      @Jean-FrançoisFabre Indeed thanks. I've fixed my answer accordingly, although it is now admittedly the same as Joe's answer.

      – blhsing
      Aug 26 '18 at 19:54











    • since it seems to be the best way, I don't see a problem. I just didn't want people copy an unefficient answer (even if it worked).

      – Jean-François Fabre
      Aug 26 '18 at 19:54

















    • that sounds overcomplicated

      – Jean-François Fabre
      Aug 26 '18 at 19:39











    • @Jean-FrançoisFabre its basically [[x,y] for key, counterValue in counterItems if counterValue == 1 for x, y in my_list if x == key]

      – Fabian N.
      Aug 26 '18 at 19:42







    • 1





      what annoys me is the double loop. The other answer has less O complexity (and also your rewrite with new variable names is clearer)

      – Jean-François Fabre
      Aug 26 '18 at 19:44






    • 1





      @Jean-FrançoisFabre Indeed thanks. I've fixed my answer accordingly, although it is now admittedly the same as Joe's answer.

      – blhsing
      Aug 26 '18 at 19:54











    • since it seems to be the best way, I don't see a problem. I just didn't want people copy an unefficient answer (even if it worked).

      – Jean-François Fabre
      Aug 26 '18 at 19:54
















    that sounds overcomplicated

    – Jean-François Fabre
    Aug 26 '18 at 19:39





    that sounds overcomplicated

    – Jean-François Fabre
    Aug 26 '18 at 19:39













    @Jean-FrançoisFabre its basically [[x,y] for key, counterValue in counterItems if counterValue == 1 for x, y in my_list if x == key]

    – Fabian N.
    Aug 26 '18 at 19:42






    @Jean-FrançoisFabre its basically [[x,y] for key, counterValue in counterItems if counterValue == 1 for x, y in my_list if x == key]

    – Fabian N.
    Aug 26 '18 at 19:42





    1




    1





    what annoys me is the double loop. The other answer has less O complexity (and also your rewrite with new variable names is clearer)

    – Jean-François Fabre
    Aug 26 '18 at 19:44





    what annoys me is the double loop. The other answer has less O complexity (and also your rewrite with new variable names is clearer)

    – Jean-François Fabre
    Aug 26 '18 at 19:44




    1




    1





    @Jean-FrançoisFabre Indeed thanks. I've fixed my answer accordingly, although it is now admittedly the same as Joe's answer.

    – blhsing
    Aug 26 '18 at 19:54





    @Jean-FrançoisFabre Indeed thanks. I've fixed my answer accordingly, although it is now admittedly the same as Joe's answer.

    – blhsing
    Aug 26 '18 at 19:54













    since it seems to be the best way, I don't see a problem. I just didn't want people copy an unefficient answer (even if it worked).

    – Jean-François Fabre
    Aug 26 '18 at 19:54





    since it seems to be the best way, I don't see a problem. I just didn't want people copy an unefficient answer (even if it worked).

    – Jean-François Fabre
    Aug 26 '18 at 19:54













    6














    An efficient solution would be to create a Counter object to hold the occurrences of the first elements, and then filter the sub-lists in the main list:



    from collections import Counter
    counts = Counter(l[0] for l in a)
    filtered = [l for l in a if counts[l[0]] == 1]
    #[[2, 4], [3, 5]]





    share|improve this answer




















    • 2





      filtered = [l for l in a if counts[l[0]] == 1] is probably clearer but yes.

      – Jean-François Fabre
      Aug 26 '18 at 19:37











    • @Jean-FrançoisFabre Fair enough. Do you understand the logic in blhsing's answer? It certainly doesn't guarantee the order of the list is maintained.

      – Joe Iddon
      Aug 26 '18 at 19:38







    • 3





      no I don't but I see a loop which makes the solution overcomplex

      – Jean-François Fabre
      Aug 26 '18 at 19:38















    6














    An efficient solution would be to create a Counter object to hold the occurrences of the first elements, and then filter the sub-lists in the main list:



    from collections import Counter
    counts = Counter(l[0] for l in a)
    filtered = [l for l in a if counts[l[0]] == 1]
    #[[2, 4], [3, 5]]





    share|improve this answer




















    • 2





      filtered = [l for l in a if counts[l[0]] == 1] is probably clearer but yes.

      – Jean-François Fabre
      Aug 26 '18 at 19:37











    • @Jean-FrançoisFabre Fair enough. Do you understand the logic in blhsing's answer? It certainly doesn't guarantee the order of the list is maintained.

      – Joe Iddon
      Aug 26 '18 at 19:38







    • 3





      no I don't but I see a loop which makes the solution overcomplex

      – Jean-François Fabre
      Aug 26 '18 at 19:38













    6












    6








    6







    An efficient solution would be to create a Counter object to hold the occurrences of the first elements, and then filter the sub-lists in the main list:



    from collections import Counter
    counts = Counter(l[0] for l in a)
    filtered = [l for l in a if counts[l[0]] == 1]
    #[[2, 4], [3, 5]]





    share|improve this answer















    An efficient solution would be to create a Counter object to hold the occurrences of the first elements, and then filter the sub-lists in the main list:



    from collections import Counter
    counts = Counter(l[0] for l in a)
    filtered = [l for l in a if counts[l[0]] == 1]
    #[[2, 4], [3, 5]]






    share|improve this answer














    share|improve this answer



    share|improve this answer








    edited Aug 26 '18 at 19:39

























    answered Aug 26 '18 at 19:30









    Joe IddonJoe Iddon

    15.2k31639




    15.2k31639







    • 2





      filtered = [l for l in a if counts[l[0]] == 1] is probably clearer but yes.

      – Jean-François Fabre
      Aug 26 '18 at 19:37











    • @Jean-FrançoisFabre Fair enough. Do you understand the logic in blhsing's answer? It certainly doesn't guarantee the order of the list is maintained.

      – Joe Iddon
      Aug 26 '18 at 19:38







    • 3





      no I don't but I see a loop which makes the solution overcomplex

      – Jean-François Fabre
      Aug 26 '18 at 19:38












    • 2





      filtered = [l for l in a if counts[l[0]] == 1] is probably clearer but yes.

      – Jean-François Fabre
      Aug 26 '18 at 19:37











    • @Jean-FrançoisFabre Fair enough. Do you understand the logic in blhsing's answer? It certainly doesn't guarantee the order of the list is maintained.

      – Joe Iddon
      Aug 26 '18 at 19:38







    • 3





      no I don't but I see a loop which makes the solution overcomplex

      – Jean-François Fabre
      Aug 26 '18 at 19:38







    2




    2





    filtered = [l for l in a if counts[l[0]] == 1] is probably clearer but yes.

    – Jean-François Fabre
    Aug 26 '18 at 19:37





    filtered = [l for l in a if counts[l[0]] == 1] is probably clearer but yes.

    – Jean-François Fabre
    Aug 26 '18 at 19:37













    @Jean-FrançoisFabre Fair enough. Do you understand the logic in blhsing's answer? It certainly doesn't guarantee the order of the list is maintained.

    – Joe Iddon
    Aug 26 '18 at 19:38






    @Jean-FrançoisFabre Fair enough. Do you understand the logic in blhsing's answer? It certainly doesn't guarantee the order of the list is maintained.

    – Joe Iddon
    Aug 26 '18 at 19:38





    3




    3





    no I don't but I see a loop which makes the solution overcomplex

    – Jean-François Fabre
    Aug 26 '18 at 19:38





    no I don't but I see a loop which makes the solution overcomplex

    – Jean-François Fabre
    Aug 26 '18 at 19:38











    6














    If you are happy to use a 3rd party library, you can use Pandas:



    import pandas as pd

    a = [[1,2],[1,0],[2,4],[3,5]]

    df = pd.DataFrame(a)
    b = df.drop_duplicates(subset=[0], keep=False).values.tolist()

    print(b)

    [[2, 4], [3, 5]]


    The trick is the keep=False argument, described in the docs for pd.DataFrame.drop_duplicates.






    share|improve this answer


















    • 3





      I never used pandas, always did things like that manually, like the other answers but THIS is another level of readability -> sparked my interest in the library -> have a +1

      – Fabian N.
      Aug 26 '18 at 19:48






    • 2





      @FabianN., I certainly don't recommend Pandas as a means of learning, but vectorised operations do have their use. And I would be disappointed to make people believe Python list + dict are the only way to structure data!

      – jpp
      Aug 26 '18 at 19:52












    • never used pandas myself, but I'm pretty sure it beats the pure python answers in terms of speed (well, don't count the import pandas part obviously :)

      – Jean-François Fabre
      Aug 26 '18 at 19:56











    • @FabianN. You should also checkout numpy - very fast, especially on large data sets like when image processing.

      – Joe Iddon
      Aug 26 '18 at 20:56












    • @JoeIddon I'm aware of numpy, I think that's the main reason I always ignored pandas. For simple things, I just used native python and for vectorized operations I used numpy so I never felt it was worth the effort to put even more layers on top... but from what I see above, with pandas, one can write code that's almost a sentence.

      – Fabian N.
      Aug 26 '18 at 21:27
















    6














    If you are happy to use a 3rd party library, you can use Pandas:



    import pandas as pd

    a = [[1,2],[1,0],[2,4],[3,5]]

    df = pd.DataFrame(a)
    b = df.drop_duplicates(subset=[0], keep=False).values.tolist()

    print(b)

    [[2, 4], [3, 5]]


    The trick is the keep=False argument, described in the docs for pd.DataFrame.drop_duplicates.






    share|improve this answer


















    • 3





      I never used pandas, always did things like that manually, like the other answers but THIS is another level of readability -> sparked my interest in the library -> have a +1

      – Fabian N.
      Aug 26 '18 at 19:48






    • 2





      @FabianN., I certainly don't recommend Pandas as a means of learning, but vectorised operations do have their use. And I would be disappointed to make people believe Python list + dict are the only way to structure data!

      – jpp
      Aug 26 '18 at 19:52












    • never used pandas myself, but I'm pretty sure it beats the pure python answers in terms of speed (well, don't count the import pandas part obviously :)

      – Jean-François Fabre
      Aug 26 '18 at 19:56











    • @FabianN. You should also checkout numpy - very fast, especially on large data sets like when image processing.

      – Joe Iddon
      Aug 26 '18 at 20:56












    • @JoeIddon I'm aware of numpy, I think that's the main reason I always ignored pandas. For simple things, I just used native python and for vectorized operations I used numpy so I never felt it was worth the effort to put even more layers on top... but from what I see above, with pandas, one can write code that's almost a sentence.

      – Fabian N.
      Aug 26 '18 at 21:27














    6












    6








    6







    If you are happy to use a 3rd party library, you can use Pandas:



    import pandas as pd

    a = [[1,2],[1,0],[2,4],[3,5]]

    df = pd.DataFrame(a)
    b = df.drop_duplicates(subset=[0], keep=False).values.tolist()

    print(b)

    [[2, 4], [3, 5]]


    The trick is the keep=False argument, described in the docs for pd.DataFrame.drop_duplicates.






    share|improve this answer













    If you are happy to use a 3rd party library, you can use Pandas:



    import pandas as pd

    a = [[1,2],[1,0],[2,4],[3,5]]

    df = pd.DataFrame(a)
    b = df.drop_duplicates(subset=[0], keep=False).values.tolist()

    print(b)

    [[2, 4], [3, 5]]


    The trick is the keep=False argument, described in the docs for pd.DataFrame.drop_duplicates.







    share|improve this answer












    share|improve this answer



    share|improve this answer










    answered Aug 26 '18 at 19:41









    jppjpp

    100k2161111




    100k2161111







    • 3





      I never used pandas, always did things like that manually, like the other answers but THIS is another level of readability -> sparked my interest in the library -> have a +1

      – Fabian N.
      Aug 26 '18 at 19:48






    • 2





      @FabianN., I certainly don't recommend Pandas as a means of learning, but vectorised operations do have their use. And I would be disappointed to make people believe Python list + dict are the only way to structure data!

      – jpp
      Aug 26 '18 at 19:52












    • never used pandas myself, but I'm pretty sure it beats the pure python answers in terms of speed (well, don't count the import pandas part obviously :)

      – Jean-François Fabre
      Aug 26 '18 at 19:56











    • @FabianN. You should also checkout numpy - very fast, especially on large data sets like when image processing.

      – Joe Iddon
      Aug 26 '18 at 20:56












    • @JoeIddon I'm aware of numpy, I think that's the main reason I always ignored pandas. For simple things, I just used native python and for vectorized operations I used numpy so I never felt it was worth the effort to put even more layers on top... but from what I see above, with pandas, one can write code that's almost a sentence.

      – Fabian N.
      Aug 26 '18 at 21:27













    • 3





      I never used pandas, always did things like that manually, like the other answers but THIS is another level of readability -> sparked my interest in the library -> have a +1

      – Fabian N.
      Aug 26 '18 at 19:48






    • 2





      @FabianN., I certainly don't recommend Pandas as a means of learning, but vectorised operations do have their use. And I would be disappointed to make people believe Python list + dict are the only way to structure data!

      – jpp
      Aug 26 '18 at 19:52












    • never used pandas myself, but I'm pretty sure it beats the pure python answers in terms of speed (well, don't count the import pandas part obviously :)

      – Jean-François Fabre
      Aug 26 '18 at 19:56











    • @FabianN. You should also checkout numpy - very fast, especially on large data sets like when image processing.

      – Joe Iddon
      Aug 26 '18 at 20:56












    • @JoeIddon I'm aware of numpy, I think that's the main reason I always ignored pandas. For simple things, I just used native python and for vectorized operations I used numpy so I never felt it was worth the effort to put even more layers on top... but from what I see above, with pandas, one can write code that's almost a sentence.

      – Fabian N.
      Aug 26 '18 at 21:27








    3




    3





    I never used pandas, always did things like that manually, like the other answers but THIS is another level of readability -> sparked my interest in the library -> have a +1

    – Fabian N.
    Aug 26 '18 at 19:48





    I never used pandas, always did things like that manually, like the other answers but THIS is another level of readability -> sparked my interest in the library -> have a +1

    – Fabian N.
    Aug 26 '18 at 19:48




    2




    2





    @FabianN., I certainly don't recommend Pandas as a means of learning, but vectorised operations do have their use. And I would be disappointed to make people believe Python list + dict are the only way to structure data!

    – jpp
    Aug 26 '18 at 19:52






    @FabianN., I certainly don't recommend Pandas as a means of learning, but vectorised operations do have their use. And I would be disappointed to make people believe Python list + dict are the only way to structure data!

    – jpp
    Aug 26 '18 at 19:52














    never used pandas myself, but I'm pretty sure it beats the pure python answers in terms of speed (well, don't count the import pandas part obviously :)

    – Jean-François Fabre
    Aug 26 '18 at 19:56





    never used pandas myself, but I'm pretty sure it beats the pure python answers in terms of speed (well, don't count the import pandas part obviously :)

    – Jean-François Fabre
    Aug 26 '18 at 19:56













    @FabianN. You should also checkout numpy - very fast, especially on large data sets like when image processing.

    – Joe Iddon
    Aug 26 '18 at 20:56






    @FabianN. You should also checkout numpy - very fast, especially on large data sets like when image processing.

    – Joe Iddon
    Aug 26 '18 at 20:56














    @JoeIddon I'm aware of numpy, I think that's the main reason I always ignored pandas. For simple things, I just used native python and for vectorized operations I used numpy so I never felt it was worth the effort to put even more layers on top... but from what I see above, with pandas, one can write code that's almost a sentence.

    – Fabian N.
    Aug 26 '18 at 21:27






    @JoeIddon I'm aware of numpy, I think that's the main reason I always ignored pandas. For simple things, I just used native python and for vectorized operations I used numpy so I never felt it was worth the effort to put even more layers on top... but from what I see above, with pandas, one can write code that's almost a sentence.

    – Fabian N.
    Aug 26 '18 at 21:27












    0














    Solution 1



    a = [[1,2],[1,0],[2,4],[3,5]]
    b =
    for item in a:
    i = 0
    if item[0] == a[i][0]:
    i =+ 1
    continue
    else:
    b.append(item)
    i += 1


    Solution 2



    a = [[1,2],[1,0],[2,4],[3,5]]
    b =

    for item in a:
    for i in range(0, len(a)):
    if item[0] == a[i][0]:
    break
    else:
    if item in b:
    continue
    else:
    b.append(item)


    Output




    (xenial)vash@localhost:~/pcc/10$ python3 remove_help.py 
    [[1, 2], [1, 0], [2, 4], [3, 5]]
    [[2, 4], [3, 5]]



    Achieved your goal no complex methods involed!
    Enjoy!






    share|improve this answer





























      0














      Solution 1



      a = [[1,2],[1,0],[2,4],[3,5]]
      b =
      for item in a:
      i = 0
      if item[0] == a[i][0]:
      i =+ 1
      continue
      else:
      b.append(item)
      i += 1


      Solution 2



      a = [[1,2],[1,0],[2,4],[3,5]]
      b =

      for item in a:
      for i in range(0, len(a)):
      if item[0] == a[i][0]:
      break
      else:
      if item in b:
      continue
      else:
      b.append(item)


      Output




      (xenial)vash@localhost:~/pcc/10$ python3 remove_help.py 
      [[1, 2], [1, 0], [2, 4], [3, 5]]
      [[2, 4], [3, 5]]



      Achieved your goal no complex methods involed!
      Enjoy!






      share|improve this answer



























        0












        0








        0







        Solution 1



        a = [[1,2],[1,0],[2,4],[3,5]]
        b =
        for item in a:
        i = 0
        if item[0] == a[i][0]:
        i =+ 1
        continue
        else:
        b.append(item)
        i += 1


        Solution 2



        a = [[1,2],[1,0],[2,4],[3,5]]
        b =

        for item in a:
        for i in range(0, len(a)):
        if item[0] == a[i][0]:
        break
        else:
        if item in b:
        continue
        else:
        b.append(item)


        Output




        (xenial)vash@localhost:~/pcc/10$ python3 remove_help.py 
        [[1, 2], [1, 0], [2, 4], [3, 5]]
        [[2, 4], [3, 5]]



        Achieved your goal no complex methods involed!
        Enjoy!






        share|improve this answer















        Solution 1



        a = [[1,2],[1,0],[2,4],[3,5]]
        b =
        for item in a:
        i = 0
        if item[0] == a[i][0]:
        i =+ 1
        continue
        else:
        b.append(item)
        i += 1


        Solution 2



        a = [[1,2],[1,0],[2,4],[3,5]]
        b =

        for item in a:
        for i in range(0, len(a)):
        if item[0] == a[i][0]:
        break
        else:
        if item in b:
        continue
        else:
        b.append(item)


        Output




        (xenial)vash@localhost:~/pcc/10$ python3 remove_help.py 
        [[1, 2], [1, 0], [2, 4], [3, 5]]
        [[2, 4], [3, 5]]



        Achieved your goal no complex methods involed!
        Enjoy!







        share|improve this answer














        share|improve this answer



        share|improve this answer








        edited Aug 26 '18 at 20:46

























        answered Aug 26 '18 at 20:35









        vash_the_stampedevash_the_stampede

        3,8291319




        3,8291319



























            draft saved

            draft discarded
















































            Thanks for contributing an answer to Stack Overflow!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid


            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.

            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f52029529%2fremoving-dupes-in-list-of-lists-in-python%23new-answer', 'question_page');

            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            𛂒𛀶,𛀽𛀑𛂀𛃧𛂓𛀙𛃆𛃑𛃷𛂟𛁡𛀢𛀟𛁤𛂽𛁕𛁪𛂟𛂯,𛁞𛂧𛀴𛁄𛁠𛁼𛂿𛀤 𛂘,𛁺𛂾𛃭𛃭𛃵𛀺,𛂣𛃍𛂖𛃶 𛀸𛃀𛂖𛁶𛁏𛁚 𛂢𛂞 𛁰𛂆𛀔,𛁸𛀽𛁓𛃋𛂇𛃧𛀧𛃣𛂐𛃇,𛂂𛃻𛃲𛁬𛃞𛀧𛃃𛀅 𛂭𛁠𛁡𛃇𛀷𛃓𛁥,𛁙𛁘𛁞𛃸𛁸𛃣𛁜,𛂛,𛃿,𛁯𛂘𛂌𛃛𛁱𛃌𛂈𛂇 𛁊𛃲,𛀕𛃴𛀜 𛀶𛂆𛀶𛃟𛂉𛀣,𛂐𛁞𛁾 𛁷𛂑𛁳𛂯𛀬𛃅,𛃶𛁼

            Edmonton

            Crossroads (UK TV series)