How to Find Duplicate Values in a Table

Duplicate values in tables can create a major problem when we do not make a primary key or a unique key on a table. In these cases, we need to find out the duplicate records and need to delete them. We can use the Having Clause to find out the duplicate records. We show this with the help of an example.

Suppose we have a table named EmployeeDeptInfo which has the column Employeid and Departmentid. The query for creating this table is given below:

Create table EmployeeDeptInfo ( Employeeid int, Departmentid int)

Employeeid contains the Id of the employee and Departmentid contains the Id of the department to which he/she belongs. This table does not have any primary key defined on it and also it doesn't have any unique key constraint defined on any column.

Suppose the table contains the following data:

Employeeid	Departmentid
1	1
2	2
3	2
4	3
3	2
2	2
5	4
2	2

In this table, entries for the employee having employeeid 2 & 3 get repeated. In this example, the data in the table is not much for the example point of view but in real time scenario, it can be billions of rows and duplication of rows can be a very big concern. Therefore, it is necessary to find out the duplicate rows in the table. We can use the Having Clause to find out the duplicate rows.

Query for finding out the duplicate rows in the table is given below:

Select Employeeid, Departmentid from EmployeeDeptInfo Group By Employeeid, 
    DepartmentId having (count(employeeid) >1 and count (Departmentid) >1)

This query will give us the following results:

Employeeid	Departmentid
2	2
3	2

We can also find out how many times a record is repeated in the table. For example, the following query gives us the result of how many times a given record is repeated.

Select Employeeid, Departmentid, 
    count (Employeeid) as NoOfRepeat from EmployeeDeptInfo Group By Employeeid, 
    DepartmentId having (count(employeeid) >1 and count (Departmentid) >1)

This query will give us the following results:

Employeeid	Departmentid	NoOfRepeat
2	2	3
3	2	2

Here, NoOfRepeat shows the number of times the records are duplicated in the table.

Summary

This article shows that we can find the duplicate records with the help of the Group By and Having Clause.