List集合常規去重與java8新特性去重方法

一、常規去重

碰到List去重的問題，除了遍歷去重，我們常常想到利用Set集合不允許重復元素的特點，通過List和Set互轉，來去掉重復元素。

// 遍歷后判斷賦給另一個list集合，保持原來順序
    public static void ridRepeat1(List<String> list) {
        System.out.println("list = [" + list + "]");
        List<String> listNew = new ArrayList<String>();
        for (String str : list) {
            if (!listNew.contains(str)) {
                listNew.add(str);
            }
        }
        System.out.println("listNew = [" + listNew + "]");
    }

    // set集合去重，保持原來順序
    public static void ridRepeat2(List<String> list) {
        System.out.println("list = [" + list + "]");
        List<String> listNew = new ArrayList<String>();
        Set set = new HashSet();
        for (String str : list) {
            if (set.add(str)) {
                listNew.add(str);
            }
        }
        System.out.println("listNew = [" + listNew + "]");
    }

    // Set去重     由於Set的無序性，不會保持原來順序
    public static void ridRepeat3(List<String> list) {
        System.out.println("list = [" + list + "]");
        Set set = new HashSet();
        List<String> listNew = new ArrayList<String>();
        set.addAll(list);
        listNew.addAll(set);
        System.out.println("listNew = [" + listNew + "]");
    }

    // Set去重（將ridRepeat3方法縮減為一行） 無序
    public static void ridRepeat4(List<String> list) {
        System.out.println("list = [" + list + "]");
        List<String> listNew = new ArrayList<String>(new HashSet(list));
        System.out.println("listNew = [" + listNew + "]");
    }

    // Set去重並保持原先順序
    public static void ridRepeat5(List<String> list) {
        System.out.println("list = [" + list + "]");
        List<String> listNew2= new ArrayList<String>(new LinkedHashSet<String>(list));
        System.out.println("listNew = [" + listNew + "]");
    }

二、java8的stream寫法實現去重

1、distinct去重

//利用java8的stream去重
  List uniqueList = list.stream().distinct().collect(Collectors.toList());
  System.out.println(uniqueList.toString());

distinct()方法默認是按照父類Object的equals與hashCode工作的。所以：

　　上面的方法在List元素為基本數據類型及String類型時是可以的，但是如果List集合元素為對象，卻不會奏效。不過如果你的實體類對象使用了目前廣泛使用的lombok插件相關注解如：@Data,那么就會自動幫你重寫了equals與hashcode方法，當然如果你的需求是根據某幾個核心字段屬性判斷去重，那么你就要在該類中自定義重寫equals與hashcode方法了。

2、也可以通過新特性簡寫方式實現

不過該方式不能保持原列表順序而是使用了TreeSet按照字典順序排序后的列表，如果需求不需要按原順序則可直接使用。

//根據name屬性去重
List<User> lt = list.stream().collect(
        collectingAndThen(
                toCollection(() -> new TreeSet<>(Comparator.comparing(User::getName))), ArrayList::new));
System.out.println("去重后的:" + lt);

//根據name與address屬性去重
List<User> lt1 = list.stream().collect(
        collectingAndThen(
                toCollection(() -> new TreeSet<>(Comparator.comparing(o -> o.getName() + ";" + o.getAddress()))), ArrayList::new));
System.out.println("去重后的:" + lt);

當需求中明確有排序要求也可以按上面簡寫方式再次加工處理使用stream流的sorted()相關API寫法。

List<User> lt = list.stream().collect(
                collectingAndThen(
                        toCollection(() -> new TreeSet<>(Comparator.comparing(User::getName))),v -> v.stream().sorted().collect(Collectors.toList())));

3、通過 filter() 方法

我們首先創建一個方法作為 Stream.filter() 的參數，其返回類型為 Predicate，原理就是判斷一個元素能否加入到 Set 中去，代碼如下：

private static <T> Predicate<T> distinctByKey(Function<? super T, ?> keyExtractor) {
    Set<Object> seen = ConcurrentHashMap.newKeySet();
    return t -> seen.add(keyExtractor.apply(t));
}

使用如下：

@Test
  public void distinctByProperty() throws JsonProcessingException {
    // 這里第二種方法我們通過過濾來實現根據對象某個屬性去重
    ObjectMapper objectMapper = new ObjectMapper();
    List<Student> studentList = getStudentList();
 
    System.out.print("去重前        :");
    System.out.println(objectMapper.writeValueAsString(studentList));
    studentList = studentList.stream().distinct().collect(Collectors.toList());
    System.out.print("distinct去重后:");
    System.out.println(objectMapper.writeValueAsString(studentList));
    // 這里我們將 distinctByKey() 方法作為 filter() 的參數，過濾掉那些不能加入到 set 的元素
    studentList = studentList.stream().filter(distinctByKey(Student::getName)).collect(Collectors.toList());
    System.out.print("根據名字去重后 :");
    System.out.println(objectMapper.writeValueAsString(studentList));
  }

去重前        :[{"stuNo":"001","name":"Tom"},{"stuNo":"001","name":"Tom"},{"stuNo":"003","name":"Tom"}]
distinct去重后:[{"stuNo":"001","name":"Tom"},{"stuNo":"003","name":"Tom"}]
根據名字去重后 :[{"stuNo":"001","name":"Tom"}]

三、相同元素累計求和等操作

　　除了集合去重意外，工作中還有一種常見的需求，例如：在所有商品訂單中，計算同一家店鋪不同商品名稱的商品成交額，可以直接通過sql語句獲取，這里寫一下如何通過java簡單實現。舉一個類似的案例：計算相同姓名與住址的用戶年齡之和。

User.java

package com.example.demo.dto;

import java.io.Serializable;
import java.util.Objects;

/**
 * @author: shf
 * description:
 * date: 2019/10/30 10:21
 */
public class User implements Serializable {

    private static final long serialVersionUID = 1L;

    private Long id;
    private String name;
    private String address;
    private Integer age;

    public User() {

    }

    public User(String name, String address, Integer age) {
        this.name = name;
        this.address = address;
        this.age = age;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public String getAddress() {
        return address;
    }

    public void setAddress(String address) {
        this.address = address;
    }

    public Integer getAge() {
        return age;
    }

    public void setAge(Integer age) {
        this.age = age;
    }

    @Override
    public String toString() {
        return "User{" +
                "name='" + name + '\'' +
                ", address='" + address + '\'' +
                ", age=" + age +
                '}';
    }

    @Override
    public boolean equals(Object obj) {
        if (this == obj) {
            return true;//地址相等
        }
        if (obj == null) {
            return false;//非空性：對於任意非空引用x，x.equals(null)應該返回false。
        }
        if (obj instanceof User) {
            User other = (User) obj;
            //需要比較的字段相等，則這兩個對象相等
            if (Objects.equals(this.name, other.name)
                    && Objects.equals(this.address, other.address)) {
                return true;
            }
        }
        return false;
    }


    @Override
    public int hashCode() {
        return Objects
                .hash(name, address);
    }
}

測試代碼：

package com.example.demo;

import com.example.demo.dto.User;

import java.util.*;
import java.util.stream.Collectors;

public class FirCes {
    public static void main(String[] args) {
        /*構建測試數據集合*/
        User user1 = new User("a小張1", "a1", 10);
        User user2 = new User("b小張2", "a2", 10);

        User user3 = new User("c小張3", "a3", 10);
        User user3_3 = new User("c小張3", "a", 10);
        User user33 = new User("c小張3", "a3", 10);

        User user4 = new User("d小張4", "a4", 10);
        User user5 = new User("e小張5", "a5", 10);
        List<User> list = new ArrayList<>();
        list.add(user1);
        list.add(user2);
        list.add(user3);
        list.add(user3_3);
        list.add(user33);
        list.add(user4);
        list.add(user5);
        //按相同name與address屬性分組User用戶
        Map<User, List<User>> listMap = list.stream().collect(Collectors.groupingBy(v -> v));
        /*先看一下分組效果*/
        listMap.forEach((key, value) -> {
            System.out.println("========");
            System.out.println("key:" + key);
            value.forEach(obj -> {
                System.out.println(obj);
            });
        });
        /*最終執行結果*/
        List<User> listNew = listMap.keySet().stream().map(u -> {
            int sum = listMap.get(u).stream().mapToInt(i -> i.getAge()).sum();
            //需要注意的是：這里也會改變原list集合中的原數據。因為這里的u分組時就是來自原集合中的一個地址對象，
            // 即：指向了原集合中的一個對象的地址。如果不想原集合被影響，這里可以new User()新的對象賦值並返回新對象
            u.setAge(sum);
            return u;
        }).collect(Collectors.toList());
        System.out.println("listNew:" + listNew);

        System.err.println("list:" + list);

        //但是一個實體類只能重寫一次equals方法，如果有多種判別需求就不好滿足了，
        // 可以定義多個不同類名相同屬性的類或者下面這種方式解決
        Map<String, List<User>> listMap1 = list.stream().collect(Collectors
                .groupingBy(v -> Optional.ofNullable(v.getName()).orElse("") + "_" + Optional.ofNullable(v.getAddress()).orElse("")));

        /*先看一下分組效果*/
        listMap1.forEach((key, value) -> {
            System.out.println("========");
            System.out.println("key:" + key);
            value.forEach(obj -> {
                System.out.println(obj);
            });
        });
        /*最終執行結果*/
        List<User> listNew1 = listMap1.keySet().stream().map(u -> {
            int sum = listMap1.get(u).stream().mapToInt(i -> i.getAge()).sum();
            User user = listMap1.get(u).get(0);
            //這里和上面一樣的原理，也會影響原list集合中的被指向的地址的對象數據
            user.setAge(sum);
            return user;
        }).collect(Collectors.toList());
        System.out.println("listNew1:" + listNew1);
        System.err.println("list:" + list);
    }
    
}

打印日志：

========
key:User{name='b小張2', address='a2', age=10}
User{name='b小張2', address='a2', age=10}
========
key:User{name='c小張3', address='a', age=10}
User{name='c小張3', address='a', age=10}
========
key:User{name='c小張3', address='a3', age=10}
User{name='c小張3', address='a3', age=10}
User{name='c小張3', address='a3', age=10}
========
key:User{name='a小張1', address='a1', age=10}
User{name='a小張1', address='a1', age=10}
========
key:User{name='d小張4', address='a4', age=10}
User{name='d小張4', address='a4', age=10}
========
key:User{name='e小張5', address='a5', age=10}
User{name='e小張5', address='a5', age=10}
listNew:[User{name='b小張2', address='a2', age=10}, User{name='c小張3', address='a', age=10}, User{name='c小張3', address='a3', age=20}, User{name='a小張1', address='a1', age=10}, User{name='d小張4', address='a4', age=10}, User{name='e小張5', address='a5', age=10}]
list:[User{name='a小張1', address='a1', age=10}, User{name='b小張2', address='a2', age=10}, User{name='c小張3', address='a3', age=20}, User{name='c小張3', address='a', age=10}, User{name='c小張3', address='a3', age=10}, User{name='d小張4', address='a4', age=10}, User{name='e小張5', address='a5', age=10}]
========
key:a小張1_a1
User{name='a小張1', address='a1', age=10}
========
key:c小張3_a
User{name='c小張3', address='a', age=10}
========
key:d小張4_a4
User{name='d小張4', address='a4', age=10}
========
key:e小張5_a5
User{name='e小張5', address='a5', age=10}
========
key:b小張2_a2
User{name='b小張2', address='a2', age=10}
========
key:c小張3_a3
User{name='c小張3', address='a3', age=20}
User{name='c小張3', address='a3', age=10}
listNew1:[User{name='a小張1', address='a1', age=10}, User{name='c小張3', address='a', age=10}, User{name='d小張4', address='a4', age=10}, User{name='e小張5', address='a5', age=10}, User{name='b小張2', address='a2', age=10}, User{name='c小張3', address='a3', age=30}]
list:[User{name='a小張1', address='a1', age=10}, User{name='b小張2', address='a2', age=10}, User{name='c小張3', address='a3', age=30}, User{name='c小張3', address='a', age=10}, User{name='c小張3', address='a3', age=10}, User{name='d小張4', address='a4', age=10}, User{name='e小張5', address='a5', age=10}]

Process finished with exit code 0

 

 

 

參考文章：

https://www.cnblogs.com/zjfjava/p/9897650.html

https://blog.csdn.net/haiyoung/article/details/80934467

https://blog.csdn.net/weixin_34185560/article/details/91464917