Cyclin-dependent kinase 2 (CDK2) drives the progression of cells into the S- and M-phases of the cell cycle. CDK2 activity is largely dispensable for normal development, but it is critically associated with tumor growth in multiple cancer types. Although the role of CDK2 in tumorigenesis has been controversial, emerging evidence proposes that selective CDK2 inhibition may provide a therapeutic benefit against certain tumors, and it continues to appeal as a strategy to exploit in anticancer drug development. Several small-molecule CDK2 inhibitors have progressed to the clinical trials. However, a CDK2-selective inhibitor is yet to be discovered. Here, we discuss the latest understandings of the role of CDK2 in normal and cancer cells, review the core pharmacophores used to target CDK2, and outline strategies for the rational design of CDK2 inhibitors. We attempt to provide an outlook on how CDK2-selective inhibitors may open new avenues for cancer therapy.