Explore the Flickr 8k dataset

Visually understanding the meaning behind expert judgements

Link to the Jupyter Notebook that originated this post.

Download the data

Download instructions available on the jbrownlee’s Datasets repo. The code here assumes the data was unzipped and is located in the flicker8k folder.

Expert judgments

Load and check the expert judgments

(5822, 5)

Check cases where all experts agree

Count the number of each score where there is an agreement between experts:

1    2350
2     580
3     214
4     247
Name: expert, dtype: int64

Load images and captions

Sample images

Auxiliary function to plot image and captions

Relevant sample (score = 4)

Irrelevant sample (score = 1)

Score = 2 (Caption describes minor aspects of the image)

Score = 3 (Caption almost describes the images with minor mistakes)