Removing dupes in list of lists in Python
Basically, I'm trying to do remove any lists that begin with the same value. For example, two of the below begin with the number 1:
a = [[1,2],[1,0],[2,4],[3,5]]
Because the value 1 exists at the start of two of the lists -- I need to remove both so that the new list becomes:
b = [[2,4],[3,5]]
How can I do this?
I've tried the below, but the output is: [[1, 2], [2, 4], [3, 5]]
def unique_by_first_n(n, coll):
seen = set()
for item in coll:
compare = tuple(item[:n])
print compare # Keep only the first `n` elements in the set
if compare not in seen:
seen.add(compare)
yield item
a = [[1,2],[1,0],[2,4],[3,5]]
filtered_list = list(unique_by_first_n(1, a))
python python-2.7 list
add a comment |
Basically, I'm trying to do remove any lists that begin with the same value. For example, two of the below begin with the number 1:
a = [[1,2],[1,0],[2,4],[3,5]]
Because the value 1 exists at the start of two of the lists -- I need to remove both so that the new list becomes:
b = [[2,4],[3,5]]
How can I do this?
I've tried the below, but the output is: [[1, 2], [2, 4], [3, 5]]
def unique_by_first_n(n, coll):
seen = set()
for item in coll:
compare = tuple(item[:n])
print compare # Keep only the first `n` elements in the set
if compare not in seen:
seen.add(compare)
yield item
a = [[1,2],[1,0],[2,4],[3,5]]
filtered_list = list(unique_by_first_n(1, a))
python python-2.7 list
2
2 also exists in both. Why don't you remove them?
– kharandziuk
Aug 26 '18 at 19:20
good point -- i only need to remove where the first item is the same. so just the 1 in this case
– Jamie Lunn
Aug 26 '18 at 19:22
add a comment |
Basically, I'm trying to do remove any lists that begin with the same value. For example, two of the below begin with the number 1:
a = [[1,2],[1,0],[2,4],[3,5]]
Because the value 1 exists at the start of two of the lists -- I need to remove both so that the new list becomes:
b = [[2,4],[3,5]]
How can I do this?
I've tried the below, but the output is: [[1, 2], [2, 4], [3, 5]]
def unique_by_first_n(n, coll):
seen = set()
for item in coll:
compare = tuple(item[:n])
print compare # Keep only the first `n` elements in the set
if compare not in seen:
seen.add(compare)
yield item
a = [[1,2],[1,0],[2,4],[3,5]]
filtered_list = list(unique_by_first_n(1, a))
python python-2.7 list
Basically, I'm trying to do remove any lists that begin with the same value. For example, two of the below begin with the number 1:
a = [[1,2],[1,0],[2,4],[3,5]]
Because the value 1 exists at the start of two of the lists -- I need to remove both so that the new list becomes:
b = [[2,4],[3,5]]
How can I do this?
I've tried the below, but the output is: [[1, 2], [2, 4], [3, 5]]
def unique_by_first_n(n, coll):
seen = set()
for item in coll:
compare = tuple(item[:n])
print compare # Keep only the first `n` elements in the set
if compare not in seen:
seen.add(compare)
yield item
a = [[1,2],[1,0],[2,4],[3,5]]
filtered_list = list(unique_by_first_n(1, a))
python python-2.7 list
python python-2.7 list
edited Aug 27 '18 at 2:16
Peter Mortensen
13.6k1985111
13.6k1985111
asked Aug 26 '18 at 19:18
Jamie LunnJamie Lunn
514
514
2
2 also exists in both. Why don't you remove them?
– kharandziuk
Aug 26 '18 at 19:20
good point -- i only need to remove where the first item is the same. so just the 1 in this case
– Jamie Lunn
Aug 26 '18 at 19:22
add a comment |
2
2 also exists in both. Why don't you remove them?
– kharandziuk
Aug 26 '18 at 19:20
good point -- i only need to remove where the first item is the same. so just the 1 in this case
– Jamie Lunn
Aug 26 '18 at 19:22
2
2
2 also exists in both. Why don't you remove them?
– kharandziuk
Aug 26 '18 at 19:20
2 also exists in both. Why don't you remove them?
– kharandziuk
Aug 26 '18 at 19:20
good point -- i only need to remove where the first item is the same. so just the 1 in this case
– Jamie Lunn
Aug 26 '18 at 19:22
good point -- i only need to remove where the first item is the same. so just the 1 in this case
– Jamie Lunn
Aug 26 '18 at 19:22
add a comment |
4 Answers
4
active
oldest
votes
You can use collections.Counter
with list comprehension to get sublists whose first item appears only once:
from collections import Counter
c = Counter(n for n, _ in a)
b = [[x, y] for x, y in a if c[x] == 1]
that sounds overcomplicated
– Jean-François Fabre
Aug 26 '18 at 19:39
@Jean-FrançoisFabre its basically[[x,y] for key, counterValue in counterItems if counterValue == 1 for x, y in my_list if x == key]
– Fabian N.
Aug 26 '18 at 19:42
1
what annoys me is the double loop. The other answer has less O complexity (and also your rewrite with new variable names is clearer)
– Jean-François Fabre
Aug 26 '18 at 19:44
1
@Jean-FrançoisFabre Indeed thanks. I've fixed my answer accordingly, although it is now admittedly the same as Joe's answer.
– blhsing
Aug 26 '18 at 19:54
since it seems to be the best way, I don't see a problem. I just didn't want people copy an unefficient answer (even if it worked).
– Jean-François Fabre
Aug 26 '18 at 19:54
add a comment |
An efficient solution would be to create a Counter
object to hold the occurrences of the first elements, and then filter the sub-lists in the main list:
from collections import Counter
counts = Counter(l[0] for l in a)
filtered = [l for l in a if counts[l[0]] == 1]
#[[2, 4], [3, 5]]
2
filtered = [l for l in a if counts[l[0]] == 1]
is probably clearer but yes.
– Jean-François Fabre
Aug 26 '18 at 19:37
@Jean-FrançoisFabre Fair enough. Do you understand the logic in blhsing's answer? It certainly doesn't guarantee the order of the list is maintained.
– Joe Iddon
Aug 26 '18 at 19:38
3
no I don't but I see a loop which makes the solution overcomplex
– Jean-François Fabre
Aug 26 '18 at 19:38
add a comment |
If you are happy to use a 3rd party library, you can use Pandas:
import pandas as pd
a = [[1,2],[1,0],[2,4],[3,5]]
df = pd.DataFrame(a)
b = df.drop_duplicates(subset=[0], keep=False).values.tolist()
print(b)
[[2, 4], [3, 5]]
The trick is the keep=False
argument, described in the docs for pd.DataFrame.drop_duplicates
.
3
I never used pandas, always did things like that manually, like the other answers but THIS is another level of readability -> sparked my interest in the library -> have a +1
– Fabian N.
Aug 26 '18 at 19:48
2
@FabianN., I certainly don't recommend Pandas as a means of learning, but vectorised operations do have their use. And I would be disappointed to make people believe Pythonlist
+dict
are the only way to structure data!
– jpp
Aug 26 '18 at 19:52
never used pandas myself, but I'm pretty sure it beats the pure python answers in terms of speed (well, don't count theimport pandas
part obviously :)
– Jean-François Fabre
Aug 26 '18 at 19:56
@FabianN. You should also checkoutnumpy
- very fast, especially on large data sets like when image processing.
– Joe Iddon
Aug 26 '18 at 20:56
@JoeIddon I'm aware of numpy, I think that's the main reason I always ignored pandas. For simple things, I just used native python and for vectorized operations I used numpy so I never felt it was worth the effort to put even more layers on top... but from what I see above, with pandas, one can write code that's almost a sentence.
– Fabian N.
Aug 26 '18 at 21:27
add a comment |
Solution 1
a = [[1,2],[1,0],[2,4],[3,5]]
b =
for item in a:
i = 0
if item[0] == a[i][0]:
i =+ 1
continue
else:
b.append(item)
i += 1
Solution 2
a = [[1,2],[1,0],[2,4],[3,5]]
b =
for item in a:
for i in range(0, len(a)):
if item[0] == a[i][0]:
break
else:
if item in b:
continue
else:
b.append(item)
Output
(xenial)vash@localhost:~/pcc/10$ python3 remove_help.py
[[1, 2], [1, 0], [2, 4], [3, 5]]
[[2, 4], [3, 5]]
Achieved your goal no complex methods involed!
Enjoy!
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f52029529%2fremoving-dupes-in-list-of-lists-in-python%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
4 Answers
4
active
oldest
votes
4 Answers
4
active
oldest
votes
active
oldest
votes
active
oldest
votes
You can use collections.Counter
with list comprehension to get sublists whose first item appears only once:
from collections import Counter
c = Counter(n for n, _ in a)
b = [[x, y] for x, y in a if c[x] == 1]
that sounds overcomplicated
– Jean-François Fabre
Aug 26 '18 at 19:39
@Jean-FrançoisFabre its basically[[x,y] for key, counterValue in counterItems if counterValue == 1 for x, y in my_list if x == key]
– Fabian N.
Aug 26 '18 at 19:42
1
what annoys me is the double loop. The other answer has less O complexity (and also your rewrite with new variable names is clearer)
– Jean-François Fabre
Aug 26 '18 at 19:44
1
@Jean-FrançoisFabre Indeed thanks. I've fixed my answer accordingly, although it is now admittedly the same as Joe's answer.
– blhsing
Aug 26 '18 at 19:54
since it seems to be the best way, I don't see a problem. I just didn't want people copy an unefficient answer (even if it worked).
– Jean-François Fabre
Aug 26 '18 at 19:54
add a comment |
You can use collections.Counter
with list comprehension to get sublists whose first item appears only once:
from collections import Counter
c = Counter(n for n, _ in a)
b = [[x, y] for x, y in a if c[x] == 1]
that sounds overcomplicated
– Jean-François Fabre
Aug 26 '18 at 19:39
@Jean-FrançoisFabre its basically[[x,y] for key, counterValue in counterItems if counterValue == 1 for x, y in my_list if x == key]
– Fabian N.
Aug 26 '18 at 19:42
1
what annoys me is the double loop. The other answer has less O complexity (and also your rewrite with new variable names is clearer)
– Jean-François Fabre
Aug 26 '18 at 19:44
1
@Jean-FrançoisFabre Indeed thanks. I've fixed my answer accordingly, although it is now admittedly the same as Joe's answer.
– blhsing
Aug 26 '18 at 19:54
since it seems to be the best way, I don't see a problem. I just didn't want people copy an unefficient answer (even if it worked).
– Jean-François Fabre
Aug 26 '18 at 19:54
add a comment |
You can use collections.Counter
with list comprehension to get sublists whose first item appears only once:
from collections import Counter
c = Counter(n for n, _ in a)
b = [[x, y] for x, y in a if c[x] == 1]
You can use collections.Counter
with list comprehension to get sublists whose first item appears only once:
from collections import Counter
c = Counter(n for n, _ in a)
b = [[x, y] for x, y in a if c[x] == 1]
edited Aug 26 '18 at 19:52
answered Aug 26 '18 at 19:27
blhsingblhsing
31.5k41336
31.5k41336
that sounds overcomplicated
– Jean-François Fabre
Aug 26 '18 at 19:39
@Jean-FrançoisFabre its basically[[x,y] for key, counterValue in counterItems if counterValue == 1 for x, y in my_list if x == key]
– Fabian N.
Aug 26 '18 at 19:42
1
what annoys me is the double loop. The other answer has less O complexity (and also your rewrite with new variable names is clearer)
– Jean-François Fabre
Aug 26 '18 at 19:44
1
@Jean-FrançoisFabre Indeed thanks. I've fixed my answer accordingly, although it is now admittedly the same as Joe's answer.
– blhsing
Aug 26 '18 at 19:54
since it seems to be the best way, I don't see a problem. I just didn't want people copy an unefficient answer (even if it worked).
– Jean-François Fabre
Aug 26 '18 at 19:54
add a comment |
that sounds overcomplicated
– Jean-François Fabre
Aug 26 '18 at 19:39
@Jean-FrançoisFabre its basically[[x,y] for key, counterValue in counterItems if counterValue == 1 for x, y in my_list if x == key]
– Fabian N.
Aug 26 '18 at 19:42
1
what annoys me is the double loop. The other answer has less O complexity (and also your rewrite with new variable names is clearer)
– Jean-François Fabre
Aug 26 '18 at 19:44
1
@Jean-FrançoisFabre Indeed thanks. I've fixed my answer accordingly, although it is now admittedly the same as Joe's answer.
– blhsing
Aug 26 '18 at 19:54
since it seems to be the best way, I don't see a problem. I just didn't want people copy an unefficient answer (even if it worked).
– Jean-François Fabre
Aug 26 '18 at 19:54
that sounds overcomplicated
– Jean-François Fabre
Aug 26 '18 at 19:39
that sounds overcomplicated
– Jean-François Fabre
Aug 26 '18 at 19:39
@Jean-FrançoisFabre its basically
[[x,y] for key, counterValue in counterItems if counterValue == 1 for x, y in my_list if x == key]
– Fabian N.
Aug 26 '18 at 19:42
@Jean-FrançoisFabre its basically
[[x,y] for key, counterValue in counterItems if counterValue == 1 for x, y in my_list if x == key]
– Fabian N.
Aug 26 '18 at 19:42
1
1
what annoys me is the double loop. The other answer has less O complexity (and also your rewrite with new variable names is clearer)
– Jean-François Fabre
Aug 26 '18 at 19:44
what annoys me is the double loop. The other answer has less O complexity (and also your rewrite with new variable names is clearer)
– Jean-François Fabre
Aug 26 '18 at 19:44
1
1
@Jean-FrançoisFabre Indeed thanks. I've fixed my answer accordingly, although it is now admittedly the same as Joe's answer.
– blhsing
Aug 26 '18 at 19:54
@Jean-FrançoisFabre Indeed thanks. I've fixed my answer accordingly, although it is now admittedly the same as Joe's answer.
– blhsing
Aug 26 '18 at 19:54
since it seems to be the best way, I don't see a problem. I just didn't want people copy an unefficient answer (even if it worked).
– Jean-François Fabre
Aug 26 '18 at 19:54
since it seems to be the best way, I don't see a problem. I just didn't want people copy an unefficient answer (even if it worked).
– Jean-François Fabre
Aug 26 '18 at 19:54
add a comment |
An efficient solution would be to create a Counter
object to hold the occurrences of the first elements, and then filter the sub-lists in the main list:
from collections import Counter
counts = Counter(l[0] for l in a)
filtered = [l for l in a if counts[l[0]] == 1]
#[[2, 4], [3, 5]]
2
filtered = [l for l in a if counts[l[0]] == 1]
is probably clearer but yes.
– Jean-François Fabre
Aug 26 '18 at 19:37
@Jean-FrançoisFabre Fair enough. Do you understand the logic in blhsing's answer? It certainly doesn't guarantee the order of the list is maintained.
– Joe Iddon
Aug 26 '18 at 19:38
3
no I don't but I see a loop which makes the solution overcomplex
– Jean-François Fabre
Aug 26 '18 at 19:38
add a comment |
An efficient solution would be to create a Counter
object to hold the occurrences of the first elements, and then filter the sub-lists in the main list:
from collections import Counter
counts = Counter(l[0] for l in a)
filtered = [l for l in a if counts[l[0]] == 1]
#[[2, 4], [3, 5]]
2
filtered = [l for l in a if counts[l[0]] == 1]
is probably clearer but yes.
– Jean-François Fabre
Aug 26 '18 at 19:37
@Jean-FrançoisFabre Fair enough. Do you understand the logic in blhsing's answer? It certainly doesn't guarantee the order of the list is maintained.
– Joe Iddon
Aug 26 '18 at 19:38
3
no I don't but I see a loop which makes the solution overcomplex
– Jean-François Fabre
Aug 26 '18 at 19:38
add a comment |
An efficient solution would be to create a Counter
object to hold the occurrences of the first elements, and then filter the sub-lists in the main list:
from collections import Counter
counts = Counter(l[0] for l in a)
filtered = [l for l in a if counts[l[0]] == 1]
#[[2, 4], [3, 5]]
An efficient solution would be to create a Counter
object to hold the occurrences of the first elements, and then filter the sub-lists in the main list:
from collections import Counter
counts = Counter(l[0] for l in a)
filtered = [l for l in a if counts[l[0]] == 1]
#[[2, 4], [3, 5]]
edited Aug 26 '18 at 19:39
answered Aug 26 '18 at 19:30
Joe IddonJoe Iddon
15.2k31639
15.2k31639
2
filtered = [l for l in a if counts[l[0]] == 1]
is probably clearer but yes.
– Jean-François Fabre
Aug 26 '18 at 19:37
@Jean-FrançoisFabre Fair enough. Do you understand the logic in blhsing's answer? It certainly doesn't guarantee the order of the list is maintained.
– Joe Iddon
Aug 26 '18 at 19:38
3
no I don't but I see a loop which makes the solution overcomplex
– Jean-François Fabre
Aug 26 '18 at 19:38
add a comment |
2
filtered = [l for l in a if counts[l[0]] == 1]
is probably clearer but yes.
– Jean-François Fabre
Aug 26 '18 at 19:37
@Jean-FrançoisFabre Fair enough. Do you understand the logic in blhsing's answer? It certainly doesn't guarantee the order of the list is maintained.
– Joe Iddon
Aug 26 '18 at 19:38
3
no I don't but I see a loop which makes the solution overcomplex
– Jean-François Fabre
Aug 26 '18 at 19:38
2
2
filtered = [l for l in a if counts[l[0]] == 1]
is probably clearer but yes.– Jean-François Fabre
Aug 26 '18 at 19:37
filtered = [l for l in a if counts[l[0]] == 1]
is probably clearer but yes.– Jean-François Fabre
Aug 26 '18 at 19:37
@Jean-FrançoisFabre Fair enough. Do you understand the logic in blhsing's answer? It certainly doesn't guarantee the order of the list is maintained.
– Joe Iddon
Aug 26 '18 at 19:38
@Jean-FrançoisFabre Fair enough. Do you understand the logic in blhsing's answer? It certainly doesn't guarantee the order of the list is maintained.
– Joe Iddon
Aug 26 '18 at 19:38
3
3
no I don't but I see a loop which makes the solution overcomplex
– Jean-François Fabre
Aug 26 '18 at 19:38
no I don't but I see a loop which makes the solution overcomplex
– Jean-François Fabre
Aug 26 '18 at 19:38
add a comment |
If you are happy to use a 3rd party library, you can use Pandas:
import pandas as pd
a = [[1,2],[1,0],[2,4],[3,5]]
df = pd.DataFrame(a)
b = df.drop_duplicates(subset=[0], keep=False).values.tolist()
print(b)
[[2, 4], [3, 5]]
The trick is the keep=False
argument, described in the docs for pd.DataFrame.drop_duplicates
.
3
I never used pandas, always did things like that manually, like the other answers but THIS is another level of readability -> sparked my interest in the library -> have a +1
– Fabian N.
Aug 26 '18 at 19:48
2
@FabianN., I certainly don't recommend Pandas as a means of learning, but vectorised operations do have their use. And I would be disappointed to make people believe Pythonlist
+dict
are the only way to structure data!
– jpp
Aug 26 '18 at 19:52
never used pandas myself, but I'm pretty sure it beats the pure python answers in terms of speed (well, don't count theimport pandas
part obviously :)
– Jean-François Fabre
Aug 26 '18 at 19:56
@FabianN. You should also checkoutnumpy
- very fast, especially on large data sets like when image processing.
– Joe Iddon
Aug 26 '18 at 20:56
@JoeIddon I'm aware of numpy, I think that's the main reason I always ignored pandas. For simple things, I just used native python and for vectorized operations I used numpy so I never felt it was worth the effort to put even more layers on top... but from what I see above, with pandas, one can write code that's almost a sentence.
– Fabian N.
Aug 26 '18 at 21:27
add a comment |
If you are happy to use a 3rd party library, you can use Pandas:
import pandas as pd
a = [[1,2],[1,0],[2,4],[3,5]]
df = pd.DataFrame(a)
b = df.drop_duplicates(subset=[0], keep=False).values.tolist()
print(b)
[[2, 4], [3, 5]]
The trick is the keep=False
argument, described in the docs for pd.DataFrame.drop_duplicates
.
3
I never used pandas, always did things like that manually, like the other answers but THIS is another level of readability -> sparked my interest in the library -> have a +1
– Fabian N.
Aug 26 '18 at 19:48
2
@FabianN., I certainly don't recommend Pandas as a means of learning, but vectorised operations do have their use. And I would be disappointed to make people believe Pythonlist
+dict
are the only way to structure data!
– jpp
Aug 26 '18 at 19:52
never used pandas myself, but I'm pretty sure it beats the pure python answers in terms of speed (well, don't count theimport pandas
part obviously :)
– Jean-François Fabre
Aug 26 '18 at 19:56
@FabianN. You should also checkoutnumpy
- very fast, especially on large data sets like when image processing.
– Joe Iddon
Aug 26 '18 at 20:56
@JoeIddon I'm aware of numpy, I think that's the main reason I always ignored pandas. For simple things, I just used native python and for vectorized operations I used numpy so I never felt it was worth the effort to put even more layers on top... but from what I see above, with pandas, one can write code that's almost a sentence.
– Fabian N.
Aug 26 '18 at 21:27
add a comment |
If you are happy to use a 3rd party library, you can use Pandas:
import pandas as pd
a = [[1,2],[1,0],[2,4],[3,5]]
df = pd.DataFrame(a)
b = df.drop_duplicates(subset=[0], keep=False).values.tolist()
print(b)
[[2, 4], [3, 5]]
The trick is the keep=False
argument, described in the docs for pd.DataFrame.drop_duplicates
.
If you are happy to use a 3rd party library, you can use Pandas:
import pandas as pd
a = [[1,2],[1,0],[2,4],[3,5]]
df = pd.DataFrame(a)
b = df.drop_duplicates(subset=[0], keep=False).values.tolist()
print(b)
[[2, 4], [3, 5]]
The trick is the keep=False
argument, described in the docs for pd.DataFrame.drop_duplicates
.
answered Aug 26 '18 at 19:41
jppjpp
100k2161111
100k2161111
3
I never used pandas, always did things like that manually, like the other answers but THIS is another level of readability -> sparked my interest in the library -> have a +1
– Fabian N.
Aug 26 '18 at 19:48
2
@FabianN., I certainly don't recommend Pandas as a means of learning, but vectorised operations do have their use. And I would be disappointed to make people believe Pythonlist
+dict
are the only way to structure data!
– jpp
Aug 26 '18 at 19:52
never used pandas myself, but I'm pretty sure it beats the pure python answers in terms of speed (well, don't count theimport pandas
part obviously :)
– Jean-François Fabre
Aug 26 '18 at 19:56
@FabianN. You should also checkoutnumpy
- very fast, especially on large data sets like when image processing.
– Joe Iddon
Aug 26 '18 at 20:56
@JoeIddon I'm aware of numpy, I think that's the main reason I always ignored pandas. For simple things, I just used native python and for vectorized operations I used numpy so I never felt it was worth the effort to put even more layers on top... but from what I see above, with pandas, one can write code that's almost a sentence.
– Fabian N.
Aug 26 '18 at 21:27
add a comment |
3
I never used pandas, always did things like that manually, like the other answers but THIS is another level of readability -> sparked my interest in the library -> have a +1
– Fabian N.
Aug 26 '18 at 19:48
2
@FabianN., I certainly don't recommend Pandas as a means of learning, but vectorised operations do have their use. And I would be disappointed to make people believe Pythonlist
+dict
are the only way to structure data!
– jpp
Aug 26 '18 at 19:52
never used pandas myself, but I'm pretty sure it beats the pure python answers in terms of speed (well, don't count theimport pandas
part obviously :)
– Jean-François Fabre
Aug 26 '18 at 19:56
@FabianN. You should also checkoutnumpy
- very fast, especially on large data sets like when image processing.
– Joe Iddon
Aug 26 '18 at 20:56
@JoeIddon I'm aware of numpy, I think that's the main reason I always ignored pandas. For simple things, I just used native python and for vectorized operations I used numpy so I never felt it was worth the effort to put even more layers on top... but from what I see above, with pandas, one can write code that's almost a sentence.
– Fabian N.
Aug 26 '18 at 21:27
3
3
I never used pandas, always did things like that manually, like the other answers but THIS is another level of readability -> sparked my interest in the library -> have a +1
– Fabian N.
Aug 26 '18 at 19:48
I never used pandas, always did things like that manually, like the other answers but THIS is another level of readability -> sparked my interest in the library -> have a +1
– Fabian N.
Aug 26 '18 at 19:48
2
2
@FabianN., I certainly don't recommend Pandas as a means of learning, but vectorised operations do have their use. And I would be disappointed to make people believe Python
list
+ dict
are the only way to structure data!– jpp
Aug 26 '18 at 19:52
@FabianN., I certainly don't recommend Pandas as a means of learning, but vectorised operations do have their use. And I would be disappointed to make people believe Python
list
+ dict
are the only way to structure data!– jpp
Aug 26 '18 at 19:52
never used pandas myself, but I'm pretty sure it beats the pure python answers in terms of speed (well, don't count the
import pandas
part obviously :)– Jean-François Fabre
Aug 26 '18 at 19:56
never used pandas myself, but I'm pretty sure it beats the pure python answers in terms of speed (well, don't count the
import pandas
part obviously :)– Jean-François Fabre
Aug 26 '18 at 19:56
@FabianN. You should also checkout
numpy
- very fast, especially on large data sets like when image processing.– Joe Iddon
Aug 26 '18 at 20:56
@FabianN. You should also checkout
numpy
- very fast, especially on large data sets like when image processing.– Joe Iddon
Aug 26 '18 at 20:56
@JoeIddon I'm aware of numpy, I think that's the main reason I always ignored pandas. For simple things, I just used native python and for vectorized operations I used numpy so I never felt it was worth the effort to put even more layers on top... but from what I see above, with pandas, one can write code that's almost a sentence.
– Fabian N.
Aug 26 '18 at 21:27
@JoeIddon I'm aware of numpy, I think that's the main reason I always ignored pandas. For simple things, I just used native python and for vectorized operations I used numpy so I never felt it was worth the effort to put even more layers on top... but from what I see above, with pandas, one can write code that's almost a sentence.
– Fabian N.
Aug 26 '18 at 21:27
add a comment |
Solution 1
a = [[1,2],[1,0],[2,4],[3,5]]
b =
for item in a:
i = 0
if item[0] == a[i][0]:
i =+ 1
continue
else:
b.append(item)
i += 1
Solution 2
a = [[1,2],[1,0],[2,4],[3,5]]
b =
for item in a:
for i in range(0, len(a)):
if item[0] == a[i][0]:
break
else:
if item in b:
continue
else:
b.append(item)
Output
(xenial)vash@localhost:~/pcc/10$ python3 remove_help.py
[[1, 2], [1, 0], [2, 4], [3, 5]]
[[2, 4], [3, 5]]
Achieved your goal no complex methods involed!
Enjoy!
add a comment |
Solution 1
a = [[1,2],[1,0],[2,4],[3,5]]
b =
for item in a:
i = 0
if item[0] == a[i][0]:
i =+ 1
continue
else:
b.append(item)
i += 1
Solution 2
a = [[1,2],[1,0],[2,4],[3,5]]
b =
for item in a:
for i in range(0, len(a)):
if item[0] == a[i][0]:
break
else:
if item in b:
continue
else:
b.append(item)
Output
(xenial)vash@localhost:~/pcc/10$ python3 remove_help.py
[[1, 2], [1, 0], [2, 4], [3, 5]]
[[2, 4], [3, 5]]
Achieved your goal no complex methods involed!
Enjoy!
add a comment |
Solution 1
a = [[1,2],[1,0],[2,4],[3,5]]
b =
for item in a:
i = 0
if item[0] == a[i][0]:
i =+ 1
continue
else:
b.append(item)
i += 1
Solution 2
a = [[1,2],[1,0],[2,4],[3,5]]
b =
for item in a:
for i in range(0, len(a)):
if item[0] == a[i][0]:
break
else:
if item in b:
continue
else:
b.append(item)
Output
(xenial)vash@localhost:~/pcc/10$ python3 remove_help.py
[[1, 2], [1, 0], [2, 4], [3, 5]]
[[2, 4], [3, 5]]
Achieved your goal no complex methods involed!
Enjoy!
Solution 1
a = [[1,2],[1,0],[2,4],[3,5]]
b =
for item in a:
i = 0
if item[0] == a[i][0]:
i =+ 1
continue
else:
b.append(item)
i += 1
Solution 2
a = [[1,2],[1,0],[2,4],[3,5]]
b =
for item in a:
for i in range(0, len(a)):
if item[0] == a[i][0]:
break
else:
if item in b:
continue
else:
b.append(item)
Output
(xenial)vash@localhost:~/pcc/10$ python3 remove_help.py
[[1, 2], [1, 0], [2, 4], [3, 5]]
[[2, 4], [3, 5]]
Achieved your goal no complex methods involed!
Enjoy!
edited Aug 26 '18 at 20:46
answered Aug 26 '18 at 20:35
vash_the_stampedevash_the_stampede
3,8291319
3,8291319
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f52029529%2fremoving-dupes-in-list-of-lists-in-python%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
2
2 also exists in both. Why don't you remove them?
– kharandziuk
Aug 26 '18 at 19:20
good point -- i only need to remove where the first item is the same. so just the 1 in this case
– Jamie Lunn
Aug 26 '18 at 19:22