Vision and Imaging Science Seminar: Sketching with Style: Visual Search with Sketches and Aesthetic Context

Speaker: John Collomosse, U. Surrey
UCL Contact: Gabriel Brostow (Visitors from outside UCL please email in advance).
Date/Time: 16 Feb 18, 11:00 - 12:00
Venue: 1.02


The internet is transforming into a visual medium; over 80% of the internet is forecast to be visual content by 2018, and most of this content will be consumed on mobile devices featuring a touch-screen as their primary interface. Gestural interaction, such as sketch, presents an intuitive way to interact with these devices. Imagine a search in which you specify your query by sketching the desired image with your finger, rather than (or in addition to) describing it with text words. In this talk, John will present recent work on the use of deep neural networks for sketch based visual search of images. He will begin by describing a scalable system for sketch based search of multi-million image databases, based upon their Gradient Field HOG (GF-HOG) descriptor. He will then describe how deep learning can be used to enhance performance of the retrieval. Imagine a product catalogue in which you sketched, say an engineering part, rather than using a text or serial numbers to find it? John will explore how sketch can be fused with other modalities such as visual style to search a large corpus of artwork e.g. the creative portfolio website Behance.Net. The talk comprises work presented previously at ICCV 2017 and ICCV 2015.

John Collomosse

Dr John Collomosse is a Reader (Assoc. Prof.) in the Centre for Vision Speech and Signal Processing (CVSSP) at the University of Surrey, and a Visiting Professor at Adobe Research within the Creative Intelligence Lab (CIL). John joined CVSSP in 2009, following 4 years lecturing at the University Bath where he also completed his PhD in Computer Vision and Graphics (2004). John has spent periods of time at IBM UK Labs, Vodafone R&D Munich, and HP Labs Bristol. John's research is cross-disciplinary, spanning Computer Vision, Computer Graphics and Artificial Intelligence, focusing on ways to search and manipulate large, unstructured iamge and video collections - to visually search media collections, and present them in aesthetic and comprehensible ways. Recent projects spanning Vision and Graphics include: sketch based search of images/video; plagiarism detection in the arts; visual search of dance; structuring and presenting large visual media collections using artistic rendering; developing characters animation from 3D multi-view capture data. John holds ~80 refereed publications, including oral presentations at ICCV, BMVC, and journal papers in IJCV, IEEE TVCG and TMM. He was general chair for NPAR 2010-11 (at SIGGRAPH), BMVC 2012, and CVMP 2014-15 and is an AE for C&G and Eurographics CGF.