Guava BloomFilter 사용 기초

사용 환경

• Windows 10
• Java 1.8
• Gradle 7.2
• Spring Boot 2.5.4
• Oracle DB 19
• MyBatis 2.2.0

프로젝트 생성하기

1. 프로젝트 생성

   브라우져에서 spring initializr를 방문하여 아래와 같이 입력하고 GENERATE 버튼을 클릭합니다.

   • Project: Gradle Project
   • Language: Java
   • Spring Boot: 2.5.5
   • Project Meta:
     • Group: trvoid.bloomfilter
     • Artifact: bloom-filter
     • Name: bloom-filter
     • Package name: trvoid.bloomfilter
     • Packaging: Jar
     • Java: 8
   • Dependencies:
     • JDBC API
     • MyBatis Framework
     • Oracle Driver

   생성된 프로젝트 파일을 다운로드하여 압축을 풉니다. 프로젝트 폴더 구조는 아래와 같습니다.

   bloom-filter
     |-src
       |-main
         |-java
           |-trvoid.bloomfilter
             |-BloomFilterApplication.java
         |-resources
           |-application.properties
       |-test
         |-java
           |-trvoid.bloomfilter
             |-BloomFilterApplicationTests.java
     |-build.gradle
   

   build.gradle 파일에서 의존성 항목을 확인할 수 있습니다.

   dependencies {
       implementation 'org.springframework.boot:spring-boot-starter-jdbc'
   	implementation 'org.mybatis.spring.boot:mybatis-spring-boot-starter:2.2.0'
       runtimeOnly 'com.oracle.database.jdbc:ojdbc8'
       testImplementation 'org.springframework.boot:spring-boot-starter-test'
   }
   

   BloomFilterApplication.java 파일의 내용은 아래와 같습니다.

   package trvoid.bloomfilter;
   
   import org.springframework.boot.SpringApplication;
   import org.springframework.boot.autoconfigure.SpringBootApplication;
   
   @SpringBootApplication
   public class BloomFilterApplication {
   
       public static void main(String[] args) {
           SpringApplication.run(BloomFilterApplication.class, args);
       }
   
   }
   

2. @Mapper 추가

   src/main/java/trvoid/bloomfilter/Reject.java

   package trvoid.bloomfilter;
   
   public class Reject {
       private String fromPhoneNo;
       private String toPhoneNo;
   
       public Reject(String fromPhoneNo, String toPhoneNo) {
           this.fromPhoneNo = fromPhoneNo;
           this.toPhoneNo = toPhoneNo;
       }
   
       public String getFromPhoneNo() {
           return fromPhoneNo;
       }
   
       public void setFromPhoneNo(String fromPhoneNo) {
           this.fromPhoneNo = fromPhoneNo;
       }
   
       public String getToPhoneNo() {
           return toPhoneNo;
       }
   
       public void setToPhoneNo(String toPhoneNo) {
           this.toPhoneNo = toPhoneNo;
       }
   
       @Override
       public String toString() {
           return String.format("Reject[fromPhoneNo:%s, toPhoneNo:%s]", fromPhoneNo, toPhoneNo);
       }
   }
   

   src/main/java/trvoid/bloomfilter/RejectMapper.java

   package trvoid.bloomfilter;
   
   import org.apache.ibatis.annotations.Insert;
   import org.apache.ibatis.annotations.Mapper;
   import org.apache.ibatis.annotations.Param;
   import org.apache.ibatis.annotations.Select;
   
   import java.util.List;
   
   @Mapper
   public interface RejectMapper {
       String SELECT = " SELECT FROM_PHONE_NO, TO_PHONE_NO FROM REJECT ";
   
       @Select(" SELECT COUNT(1) FROM ALL_OBJECTS " +
               " WHERE OBJECT_TYPE = 'TABLE' AND OBJECT_NAME = 'REJECT' ")
       int countTable();
   
       @Insert(" CREATE TABLE REJECT ( " +
               "    FROM_PHONE_NO VARCHAR2(20) NOT NULL, " +
               "    TO_PHONE_NO VARCHAR2(20) NOT NULL, " +
               "    CONSTRAINT REJECT_PK PRIMARY KEY (FROM_PHONE_NO, TO_PHONE_NO) " +
               " ) ")
       void createTable();
   
       @Insert(" INSERT INTO REJECT ( " +
               "    FROM_PHONE_NO, TO_PHONE_NO " +
               " ) VALUES ( " +
               "    #{reject.fromPhoneNo}, #{reject.toPhoneNo} " +
               " ) ")
       void insertReject(@Param("reject") Reject reject);
   
       @Select("SELECT COUNT(*) FROM REJECT WHERE FROM_PHONE_NO = #{fromPhoneNo} AND TO_PHONE_NO = #{toPhoneNo} ")
       int countCar(@Param("fromPhoneNo") String fromPhoneNo, @Param("toPhoneNo") String toPhoneNo);
   
       @Select(SELECT)
       List<Reject> findAll();
   }
   

3. 데이터베이스 질의 수행을 위해 CommandLineRunner 추가

   REJECT 테이블이 없을 경우 생성합니다.

   src/main/java/trvoid/bloomfilter/BloomFilterApplication.java

   package trvoid.bloomfilter;
   
   import org.springframework.beans.factory.annotation.Autowired;
   import org.springframework.boot.CommandLineRunner;
   import org.springframework.boot.SpringApplication;
   import org.springframework.boot.autoconfigure.SpringBootApplication;
   
   import javax.sql.DataSource;
   
   @SpringBootApplication
   public class BloomFilterApplication implements CommandLineRunner {
       @Autowired
       DataSource dataSource;
   
       @Autowired
       RejectMapper rejectMapper;
   
       public static void main(String[] args) {
           SpringApplication.run(BloomFilterApplication.class, args);
       }
       
       @Override
       public void run(String... args) throws Exception {
           System.out.println("DataSource = " + dataSource);
   
           if (rejectMapper.countTable() == 0) {
               rejectMapper.createTable();
               System.out.println("** Created a table: REJECT");
           }
           
           System.exit(0);
       }
   }
   

4. 데이터 소스 설정

   src/main/resources/application.properties

   spring.datasource.url=jdbc:oracle:thin:@localhost:1521:orcl
   spring.datasource.username=your_db_username
   spring.datasource.password=your_db_password
   spring.datasource.driver-class-name=oracle.jdbc.OracleDriver
   

5. 실행

   >gradlew bootRun
   

   출력 결과

   ...
   Create a Bloom Filter instance.
   Add all the REJECT data.
   Test the Bloom Filter.
   0800000000101000000001 -> true
   0800000000102000000001 -> false
   ...
   

블룸필터 적용하기

1. Guava 라이브러리에 대한 의존성 추가

   build.gradle

   implementation 'com.google.guava:guava:31.0.1-jre'
   

2. 블룸필터 생성 및 테스트 메쏘드 추가

   src/main/java/trvoid/bloomfilter/BloomFilterApplication.java

   private BloomFilter<String> initBloomFilter() {
   	System.out.println("Create a Bloom Filter instance.");
   	BloomFilter<String> rejectFilter = BloomFilter.create(Funnels.stringFunnel(Charset.forName("UTF-8")),100000);
   
   	System.out.println("Add all the REJECT data.");
   	List<Reject> list = rejectMapper.findAll();
   	list.forEach(x -> rejectFilter.put(x.getFromPhoneNo() + x.getToPhoneNo()));
   
   	return rejectFilter;
   }
   
   private void testBloomFilter(BloomFilter<String> rejectFilter) {
   	System.out.println("Test the Bloom Filter.");
   	String[] values = new String[] {
   		"0800000000101000000001",
   		"0800000000102000000001"
   	};
   
   	for (String s : values) {
   		System.out.println(String.format("%s -> %s", s, rejectFilter.mightContain(s)));
   	}
   }
   

3. 블룸필터 생성 및 테스트 메쏘드 호출

   src/main/java/trvoid/bloomfilter/BloomFilterApplication.java

   @Override
   public void run(String... args) throws Exception {
   	System.out.println("DataSource = " + dataSource);
   
   	if (rejectMapper.countTable() == 0) {
   		rejectMapper.createTable();
   		System.out.println("** Created a table: REJECT");
   	}
   
   	//insertTestData();
   
   	BloomFilter<String> rejectFilter = initBloomFilter();
   	testBloomFilter(rejectFilter);
   
   	System.exit(0);
   }
   

Written with StackEdit.