A content-based image retrieval (CBIR) system has a front-end that includes a pipeline of one or more dynamically-constructed filters for measuring perceptual similarities between a query image and one or more candidate images retrieved from a back-end comprised of a knowledge base accessed by an inference engine. The images include at least one color set having a set of properties including a number of pixels each having at least one color, a culture color associated with the color set, a mean and variance of the color set, a moment invariant, and a centroid. The filters analyze and compare the set of properties of the query image to the set of properties of the candidate images. Various filters are used, including: a Color Mask filter that identifies identical culture colors in the images, a Color Histogram filter that identifies a distribution of colors in the images, a Color Average filter that performs a similarity comparison on the average of the color sets of the images, a Color Variance filter that performs a similarity comparison on the variances of the color sets of the images, a Spread filter that identifies a spatial concentration of a color in the images, an Elongation filter that identifies a shape of a color in the images, and a Spatial Relationship filter that identifies a spatial relationship between the color sets in the images.