首页 > 最新文献
paperpass

Proceedings of the 19th ACM international conference on Multimedia最新文献

英文 中文
A mobile location search system with active query sensing 一种具有主动查询感知的移动位置搜索系统
Pub Date : 2011-11-28 DOI: 10.1145/2072298.2072468
Felix X. Yu, R. Ji, Tongtao Zhang, Shih-Fu Chang
How should the second query be taken once the first query fails in mobile location search based on visual recognition? In this demo, we describe a mobile search system with a unique Active Query Sensing (AQS) function to intelligently guide the mobile user to take a successful second query. This suggestion is built upon a scalable visual matching system covering over 0.3 million street view reference images in New York City, where each location is associated with multiple surrounding views and panorama. In online search, once the initial search result fails, the system will perform online analysis and suggest the mobile user to turn to the most discriminative viewing angle to take the second visual query, from which the search performance is expected to greatly improve. The AQS suggestion is based on both offline salient view discovery and online viewing angle prediction and intelligent turning decision. Our experiments show our AVS can improve the mobile location search with a performance gain as high as 100%, reducing the failure rate to only 12% after taking the second visual query.
在基于视觉识别的移动位置搜索中,一旦第一个查询失败,第二个查询应该如何进行?在这个演示中,我们描述了一个具有独特的主动查询感知(AQS)功能的移动搜索系统,该功能可以智能地引导移动用户进行成功的第二次查询。这个建议是建立在一个可扩展的视觉匹配系统上的,该系统覆盖了纽约市超过30万张街景参考图像,其中每个位置都与多个周围视图和全景相关联。在在线搜索中,一旦初始搜索结果失败,系统将进行在线分析,并建议移动用户转向最具判别性的视角进行第二次视觉查询,从而有望大大提高搜索性能。AQS建议是基于离线突出视图发现和在线视角预测和智能转弯决策。我们的实验表明,我们的AVS可以提高移动位置搜索的性能,性能提升高达100%,在进行第二次视觉查询后,将故障率降低到仅12%。
{"title":"A mobile location search system with active query sensing","authors":"Felix X. Yu, R. Ji, Tongtao Zhang, Shih-Fu Chang","doi":"10.1145/2072298.2072468","DOIUrl":"https://doi.org/10.1145/2072298.2072468","url":null,"abstract":"How should the second query be taken once the first query fails in mobile location search based on visual recognition? In this demo, we describe a mobile search system with a unique Active Query Sensing (AQS) function to intelligently guide the mobile user to take a successful second query. This suggestion is built upon a scalable visual matching system covering over 0.3 million street view reference images in New York City, where each location is associated with multiple surrounding views and panorama. In online search, once the initial search result fails, the system will perform online analysis and suggest the mobile user to turn to the most discriminative viewing angle to take the second visual query, from which the search performance is expected to greatly improve. The AQS suggestion is based on both offline salient view discovery and online viewing angle prediction and intelligent turning decision. Our experiments show our AVS can improve the mobile location search with a performance gain as high as 100%, reducing the failure rate to only 12% after taking the second visual query.","PeriodicalId":318758,"journal":{"name":"Proceedings of the 19th ACM international conference on Multimedia","volume":"87 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121096990","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
Flicker effects in adaptive video streaming to handheld devices 自适应视频流到手持设备中的闪烁效果
Pub Date : 2011-11-28 DOI: 10.1145/2072298.2072359
Pengpeng Ni, Ragnhild Eg, Alexander Eichhorn, C. Griwodz, P. Halvorsen
Streaming video over the Internet requires mechanisms that limit the streams' bandwidth consumption within its fair share. TCP streaming guarantees this and provides lossless streaming as a side-effect. Adaptation by packet drop does not occur in the network, and excessive startup latency and stalling must be prevented by adapting the bandwidth consumption of the video itself. However, when the adaptation is performed during an ongoing session, it may influence the perceived quality of the entire video and result in improved or reduced visual quality of experience. We have investigated visual artifacts that are caused by adaptive layer switching -- we call them flicker effects -- and present our results for handheld devices in this paper. We considered three types of flicker, namely noise, blur and motion flicker. The perceptual impact of flicker is explored through subjective assessments. We vary both the intensity of quality changes (amplitude) and the number of quality changes per second (frequency). Users' ability to detect and their acceptance of variations in the amplitudes and frequencies of the quality changes are explored across four content types. Our results indicate that multiple factors influence the acceptance of different quality variations. Amplitude plays the dominant role in delivering satisfactory video quality, while frequency can also be adjusted to relieve the annoyance of flicker artifacts.
互联网上的流视频需要一种机制,将流的带宽消耗限制在公平的份额之内。TCP流保证了这一点,并提供了无损流作为副作用。网络中不会出现丢包的适应,必须通过调整视频本身的带宽消耗来防止过度的启动延迟和延迟。但是,如果在正在进行的会议期间进行调整,则可能影响整个视频的感知质量,并导致视觉体验质量的改善或降低。我们研究了由自适应层切换引起的视觉伪影——我们称之为闪烁效应——并在本文中展示了我们在手持设备上的结果。我们考虑了三种类型的闪烁,即噪声、模糊和运动闪烁。通过主观评价来探讨闪烁对感知的影响。我们改变质量变化的强度(振幅)和每秒质量变化的次数(频率)。通过四种内容类型探讨用户检测和接受质量变化幅度和频率变化的能力。我们的研究结果表明,多种因素影响不同质量变化的接受程度。振幅在提供令人满意的视频质量方面起着主导作用,而频率也可以调整以减轻闪烁伪影的烦恼。
{"title":"Flicker effects in adaptive video streaming to handheld devices","authors":"Pengpeng Ni, Ragnhild Eg, Alexander Eichhorn, C. Griwodz, P. Halvorsen","doi":"10.1145/2072298.2072359","DOIUrl":"https://doi.org/10.1145/2072298.2072359","url":null,"abstract":"Streaming video over the Internet requires mechanisms that limit the streams' bandwidth consumption within its fair share. TCP streaming guarantees this and provides lossless streaming as a side-effect. Adaptation by packet drop does not occur in the network, and excessive startup latency and stalling must be prevented by adapting the bandwidth consumption of the video itself. However, when the adaptation is performed during an ongoing session, it may influence the perceived quality of the entire video and result in improved or reduced visual quality of experience. We have investigated visual artifacts that are caused by adaptive layer switching -- we call them flicker effects -- and present our results for handheld devices in this paper. We considered three types of flicker, namely noise, blur and motion flicker. The perceptual impact of flicker is explored through subjective assessments. We vary both the intensity of quality changes (amplitude) and the number of quality changes per second (frequency). Users' ability to detect and their acceptance of variations in the amplitudes and frequencies of the quality changes are explored across four content types. Our results indicate that multiple factors influence the acceptance of different quality variations. Amplitude plays the dominant role in delivering satisfactory video quality, while frequency can also be adjusted to relieve the annoyance of flicker artifacts.","PeriodicalId":318758,"journal":{"name":"Proceedings of the 19th ACM international conference on Multimedia","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122872130","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 128
Semi-supervised face image retrieval using sparse coding with identity constraint 基于身份约束的稀疏编码半监督人脸图像检索
Pub Date : 2011-11-28 DOI: 10.1145/2072298.2072017
Bor-Chun Chen, Y. Kuo, Yan-Ying Chen, Kuan-Yu Chu, Winston H. Hsu
We aim to develop a scalable face image retrieval system which can integrate with partial identity information to improve the retrieval result. To achieve this goal, we first apply sparse coding on local features extracted from face images combining with inverted indexing to construct an efficient and scalable face retrieval system. We then propose a novel coding scheme that refines the representation of the original sparse coding by using identity information. Using the proposed coding scheme, face images with large intra-class variances will still be quantized into similar visual words if they share the same identity. Experimental results show that our system can achieve salient retrieval results on LFW dataset (13K faces) and outperform linear search methods using well known face recognition feature descriptors.
我们的目标是开发一个可扩展的人脸图像检索系统,该系统可以集成部分身份信息,以提高检索结果。为了实现这一目标,我们首先对从人脸图像中提取的局部特征进行稀疏编码,并结合倒排索引构建一个高效、可扩展的人脸检索系统。然后,我们提出了一种新的编码方案,通过使用身份信息来改进原始稀疏编码的表示。使用所提出的编码方案,类内方差较大的人脸图像在具有相同身份的情况下仍将被量化为相似的视觉词。实验结果表明,该系统可以在LFW数据集(13K张人脸)上取得显著的检索结果,并且优于使用已知人脸识别特征描述符的线性搜索方法。
{"title":"Semi-supervised face image retrieval using sparse coding with identity constraint","authors":"Bor-Chun Chen, Y. Kuo, Yan-Ying Chen, Kuan-Yu Chu, Winston H. Hsu","doi":"10.1145/2072298.2072017","DOIUrl":"https://doi.org/10.1145/2072298.2072017","url":null,"abstract":"We aim to develop a scalable face image retrieval system which can integrate with partial identity information to improve the retrieval result. To achieve this goal, we first apply sparse coding on local features extracted from face images combining with inverted indexing to construct an efficient and scalable face retrieval system. We then propose a novel coding scheme that refines the representation of the original sparse coding by using identity information. Using the proposed coding scheme, face images with large intra-class variances will still be quantized into similar visual words if they share the same identity. Experimental results show that our system can achieve salient retrieval results on LFW dataset (13K faces) and outperform linear search methods using well known face recognition feature descriptors.","PeriodicalId":318758,"journal":{"name":"Proceedings of the 19th ACM international conference on Multimedia","volume":"11 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126699759","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 45
Eye-tracking methodology and applications to images and video 眼动追踪方法及其在图像和视频中的应用
Pub Date : 2011-11-28 DOI: 10.1145/2072298.2072406
Harish Katti, M. Kankanhalli
Our tutorial introduces eye-tracking as an exciting, non-intrusive method of capturing user attention during human interaction with digital images and videos. We believe eye-gaze can play a valuable role in understanding and processing (a) huge volumes of image and video content generated as a result of human experiences and interaction with the environment (b) Personalization and in human-media interaction, having access to individual preferences and behavioral patterns would be a key component of such a system. Recent possibilities to seamlessly integrate eye-tracking into laptops and mobile devices opens up a plethora of possibilities for applications that can respond to user's visual attention strategies. (c) Visual content design such as in advertising often employs techniques that guide user attention to produce visual impact and elements of surprise and emotion. Eye-gaze has been used as a tool to evaluate different choices of visual elements and their placement. (d) Affective analysis of images and videos is an ongoing and challenging area in multimedia research, show recent results on how eye-gaze and accompanying pupillary dilation information can aid affective analysis.
我们的教程介绍了眼动追踪作为一种令人兴奋的、非侵入性的方法,在人类与数字图像和视频的互动中捕捉用户的注意力。我们相信眼睛的注视可以在理解和处理(a)由于人类经验和与环境的互动而产生的大量图像和视频内容(b)个性化和在人与媒体互动中,获得个人偏好和行为模式将是这样一个系统的关键组成部分。最近将眼球追踪无缝集成到笔记本电脑和移动设备的可能性,为响应用户视觉注意力策略的应用程序提供了大量的可能性。(c)广告等视觉内容设计经常采用引导用户注意的技术,以产生视觉冲击和惊喜和情感元素。眼睛凝视被用作评估视觉元素及其位置的不同选择的工具。(d)图像和录像的情感分析是多媒体研究中一个正在进行和具有挑战性的领域,显示关于眼睛注视和伴随的瞳孔扩张信息如何有助于情感分析的最新结果。
{"title":"Eye-tracking methodology and applications to images and video","authors":"Harish Katti, M. Kankanhalli","doi":"10.1145/2072298.2072406","DOIUrl":"https://doi.org/10.1145/2072298.2072406","url":null,"abstract":"Our tutorial introduces eye-tracking as an exciting, non-intrusive method of capturing user attention during human interaction with digital images and videos. We believe eye-gaze can play a valuable role in understanding and processing (a) huge volumes of image and video content generated as a result of human experiences and interaction with the environment (b) Personalization and in human-media interaction, having access to individual preferences and behavioral patterns would be a key component of such a system. Recent possibilities to seamlessly integrate eye-tracking into laptops and mobile devices opens up a plethora of possibilities for applications that can respond to user's visual attention strategies. (c) Visual content design such as in advertising often employs techniques that guide user attention to produce visual impact and elements of surprise and emotion. Eye-gaze has been used as a tool to evaluate different choices of visual elements and their placement. (d) Affective analysis of images and videos is an ongoing and challenging area in multimedia research, show recent results on how eye-gaze and accompanying pupillary dilation information can aid affective analysis.","PeriodicalId":318758,"journal":{"name":"Proceedings of the 19th ACM international conference on Multimedia","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131501354","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
SRV-TaGS: An Automatic TAGging and Search System for Sensor-Rich Outdoor Videos SRV-TaGS:一个传感器丰富的户外视频自动标记和搜索系统
Pub Date : 2011-11-28 DOI: 10.1145/2072298.2072444
Zhijie Shen, Sakire Arslan Ay, S. H. Kim
Tagging facilitates video search in many social media and web applications. While manual tagging is time consuming, subjective and sometimes inaccurate, auto-tagging facilitated by content-based techniques is compute-intensive and challenging to apply across domains. We have developed a complementary system, named SRV-TAGS, to automatically generate tags for outdoor videos based on their geographic properties, to index the videos based on their generated tags and to provide textual search services. The system works with our geo-referenced video management web portal, enabling users to manage, search and watch videos.
在许多社交媒体和web应用程序中,标记有助于视频搜索。手动标记耗时、主观且有时不准确,而基于内容的技术促进的自动标记是计算密集型的,跨领域应用具有挑战性。我们开发了一个互补的系统,名为SRV-TAGS,可以根据户外视频的地理属性自动生成标签,根据生成的标签对视频进行索引,并提供文本搜索服务。该系统与我们的地理参考视频管理门户网站一起工作,使用户能够管理,搜索和观看视频。
{"title":"SRV-TaGS: An Automatic TAGging and Search System for Sensor-Rich Outdoor Videos","authors":"Zhijie Shen, Sakire Arslan Ay, S. H. Kim","doi":"10.1145/2072298.2072444","DOIUrl":"https://doi.org/10.1145/2072298.2072444","url":null,"abstract":"Tagging facilitates video search in many social media and web applications. While manual tagging is time consuming, subjective and sometimes inaccurate, auto-tagging facilitated by content-based techniques is compute-intensive and challenging to apply across domains. We have developed a complementary system, named SRV-TAGS, to automatically generate tags for outdoor videos based on their geographic properties, to index the videos based on their generated tags and to provide textual search services. The system works with our geo-referenced video management web portal, enabling users to manage, search and watch videos.","PeriodicalId":318758,"journal":{"name":"Proceedings of the 19th ACM international conference on Multimedia","volume":"222 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122405372","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Contextual synonym dictionary for visual object retrieval 上下文同义词字典的视觉对象检索
Pub Date : 2011-11-28 DOI: 10.1145/2072298.2072364
Wenbin Tang, Rui Cai, Zhiwei Li, Lei Zhang
In this paper, we study the problem of visual object retrieval by introducing a dictionary of contextual synonyms to narrow down the semantic gap in visual word quantization. The basic idea is to expand a visual word in the query image with its synonyms to boost the retrieval recall. Unlike the existing work such as soft-quantization, which only focuses on the Euclidean (l2) distance in descriptor space, we utilize the visual words which are more likely to describe visual objects with the same semantic meaning by identifying the words with similar contextual distributions (i.e. contextual synonyms). We describe the contextual distribution of a visual word using the statistics of both co-occurrence and spatial information averaged over all the image patches having this visual word, and propose an efficient system implementation to construct the contextual synonym dictionary for a large visual vocabulary. The whole construction process is unsupervised and the synonym dictionary can be naturally integrated into a standard bag-of-feature image retrieval system. Experimental results on several benchmark datasets are quite promising. The contextual synonym dictionary-based expansion consistently outperforms the l2 distance-based soft-quantization, and advances the state-of-the-art performance remarkably.
本文通过引入上下文同义词词典来缩小视觉词量化中的语义差距,研究了视觉对象检索问题。其基本思想是将查询图像中的视觉词与其同义词展开,以提高检索召回率。与软量化等现有工作只关注描述符空间中的欧氏距离(l2)不同,我们通过识别具有相似上下文分布(即上下文同义词)的词来利用更可能描述具有相同语义的视觉对象的视觉词。本文通过统计具有该视觉词的所有图像补丁的共现信息和空间信息的平均值来描述视觉词的上下文分布,并提出了一种有效的系统实现,用于构建大型视觉词汇的上下文同义词词典。整个构建过程是无监督的,同义词词典可以自然地集成到标准的特征袋图像检索系统中。在几个基准数据集上的实验结果很有希望。基于上下文同义词字典的扩展始终优于基于l2距离的软量化,并显著提高了最先进的性能。
{"title":"Contextual synonym dictionary for visual object retrieval","authors":"Wenbin Tang, Rui Cai, Zhiwei Li, Lei Zhang","doi":"10.1145/2072298.2072364","DOIUrl":"https://doi.org/10.1145/2072298.2072364","url":null,"abstract":"In this paper, we study the problem of visual object retrieval by introducing a dictionary of contextual synonyms to narrow down the semantic gap in visual word quantization. The basic idea is to expand a visual word in the query image with its synonyms to boost the retrieval recall. Unlike the existing work such as soft-quantization, which only focuses on the Euclidean (l2) distance in descriptor space, we utilize the visual words which are more likely to describe visual objects with the same semantic meaning by identifying the words with similar contextual distributions (i.e. contextual synonyms). We describe the contextual distribution of a visual word using the statistics of both co-occurrence and spatial information averaged over all the image patches having this visual word, and propose an efficient system implementation to construct the contextual synonym dictionary for a large visual vocabulary. The whole construction process is unsupervised and the synonym dictionary can be naturally integrated into a standard bag-of-feature image retrieval system. Experimental results on several benchmark datasets are quite promising. The contextual synonym dictionary-based expansion consistently outperforms the l2 distance-based soft-quantization, and advances the state-of-the-art performance remarkably.","PeriodicalId":318758,"journal":{"name":"Proceedings of the 19th ACM international conference on Multimedia","volume":"5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127709452","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 21
The FCam API for programmable cameras 用于可编程相机的FCam API
Pub Date : 2011-11-28 DOI: 10.1145/2072298.2072425
S. H. Park, Andrew Adams, Eino-Ville Talvala
The FCam API is an open-source camera control library, enabling precise control over a camera's imaging pipeline. Intended for researchers and students in the field of computational photography, it allows easy implementation of novel algorithms and applications. Currently implemented on the Nokia N900 smartphone, and a custom-built "Frankencamera", it has been used in teaching at universities around the world, and is freely available for download for the N900. This paper describes the architecture underlying the API, the design of the API itself, several applications built on top of it, and some examples of its use in education.
FCam API是一个开源相机控制库,可以对相机的成像管道进行精确控制。它适用于计算摄影领域的研究人员和学生,可以轻松实现新的算法和应用程序。目前在诺基亚N900智能手机和定制的“弗兰肯相机”上实现,它已在世界各地的大学教学中使用,并且可以免费下载N900。本文描述了API的底层架构、API本身的设计、在此基础上构建的几个应用程序,以及在教育中使用API的一些示例。
{"title":"The FCam API for programmable cameras","authors":"S. H. Park, Andrew Adams, Eino-Ville Talvala","doi":"10.1145/2072298.2072425","DOIUrl":"https://doi.org/10.1145/2072298.2072425","url":null,"abstract":"The FCam API is an open-source camera control library, enabling precise control over a camera's imaging pipeline. Intended for researchers and students in the field of computational photography, it allows easy implementation of novel algorithms and applications. Currently implemented on the Nokia N900 smartphone, and a custom-built \"Frankencamera\", it has been used in teaching at universities around the world, and is freely available for download for the N900. This paper describes the architecture underlying the API, the design of the API itself, several applications built on top of it, and some examples of its use in education.","PeriodicalId":318758,"journal":{"name":"Proceedings of the 19th ACM international conference on Multimedia","volume":"23 1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121423730","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
Extracting intentionally captured regions using point trajectories 使用点轨迹提取有意捕获的区域
Pub Date : 2011-11-28 DOI: 10.1145/2072298.2072029
Yuta Nakashima, N. Babaguchi
When camera persons take videos with mobile video cameras, they usually have capture intentions, i.e., what they want to express in their videos, and there are intentionally captured regions (ICRs) in the video frames that are essential for the capture intentions. Extracting ICRs is thus beneficial for wide range of applications such as video summarization and video adaptation for small displays. In this paper, we present a novel method for automatically extracting ICRs. A camera person usually moves his/her camera so that ICRs can be arranged in appropriate positions in video frames; therefore, ICRs can yield specific motion. This observation indicates that such specific motion is a vital cue for extracting ICRs. The proposed method represents motion by point trajectories, which are long-term trajectories of spatially dense points in video frames, and extracts ICRs using an ICR model based on the point trajectories. We experimentally evaluate the proposed method to demonstrate its potential applicability.
当摄影师使用移动摄像机拍摄视频时,他们通常有捕捉意图,即他们想要在视频中表达什么,并且在视频帧中存在对捕捉意图至关重要的有意捕获区域(ICRs)。因此,ICRs的提取有利于视频摘要和小显示器视频自适应等广泛的应用。本文提出了一种自动提取ICRs的新方法。摄像人员通常会移动他/她的摄像机,以便将icr安排在视频帧中的适当位置;因此,icr可以产生特定的运动。这一观察结果表明,这种特定的运动是提取icr的重要线索。该方法通过点轨迹(视频帧中空间密集点的长期轨迹)表示运动,并利用基于点轨迹的ICR模型提取ICR。我们对所提出的方法进行了实验评估,以证明其潜在的适用性。
{"title":"Extracting intentionally captured regions using point trajectories","authors":"Yuta Nakashima, N. Babaguchi","doi":"10.1145/2072298.2072029","DOIUrl":"https://doi.org/10.1145/2072298.2072029","url":null,"abstract":"When camera persons take videos with mobile video cameras, they usually have capture intentions, i.e., what they want to express in their videos, and there are intentionally captured regions (ICRs) in the video frames that are essential for the capture intentions. Extracting ICRs is thus beneficial for wide range of applications such as video summarization and video adaptation for small displays. In this paper, we present a novel method for automatically extracting ICRs. A camera person usually moves his/her camera so that ICRs can be arranged in appropriate positions in video frames; therefore, ICRs can yield specific motion. This observation indicates that such specific motion is a vital cue for extracting ICRs. The proposed method represents motion by point trajectories, which are long-term trajectories of spatially dense points in video frames, and extracts ICRs using an ICR model based on the point trajectories. We experimentally evaluate the proposed method to demonstrate its potential applicability.","PeriodicalId":318758,"journal":{"name":"Proceedings of the 19th ACM international conference on Multimedia","volume":"17 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129389797","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
News contextualization with geographic and visual information 新闻语境化与地理和视觉信息
Pub Date : 2011-11-28 DOI: 10.1145/2072298.2072317
Zechao Li, M. Wang, J. Liu, Changsheng Xu, Hanqing Lu
In this paper, we investigate the contextualization of news documents with geographic and visual information. We propose a matrix factorization approach to analyze the location relevance for each news document. We also propose a method to enrich the document with a set of web images. For location relevance analysis, we first perform toponym extraction and expansion to obtain a toponym list from news documents. We then propose a matrix factorization method to estimate the location-document relevance scores while simultaneously capturing the correlation of locations and documents. For image enrichment, we propose a method to generate multiple queries from each news document for image search and then employ an intelligent fusion approach to collect a set of images from the search results. Based on the location relevance analysis and image enrichment, we introduce a news browsing system named NewsMap which can support users in reading news via browsing a map and retrieving news with location queries. The news documents with the corresponding enriched images are presented to help users quickly get information. Extensive experiments demonstrate the effectiveness of our approaches.
在本文中,我们研究新闻文件的语境化与地理和视觉信息。我们提出了一种矩阵分解方法来分析每个新闻文档的位置相关性。我们还提出了一种用一组web图像来丰富文档的方法。对于位置相关性分析,我们首先进行地名提取和扩展,从新闻文档中获得地名列表。然后,我们提出了一种矩阵分解方法来估计位置-文档相关性分数,同时捕获位置和文档的相关性。对于图像丰富,我们提出了一种从每个新闻文档生成多个查询用于图像搜索的方法,然后采用智能融合方法从搜索结果中收集一组图像。基于位置相关性分析和图像丰富,我们介绍了一个新闻浏览系统NewsMap,它可以支持用户通过浏览地图来阅读新闻,并通过位置查询来检索新闻。新闻文档与相应的丰富的图像,以帮助用户快速获取信息。大量的实验证明了我们方法的有效性。
{"title":"News contextualization with geographic and visual information","authors":"Zechao Li, M. Wang, J. Liu, Changsheng Xu, Hanqing Lu","doi":"10.1145/2072298.2072317","DOIUrl":"https://doi.org/10.1145/2072298.2072317","url":null,"abstract":"In this paper, we investigate the contextualization of news documents with geographic and visual information. We propose a matrix factorization approach to analyze the location relevance for each news document. We also propose a method to enrich the document with a set of web images. For location relevance analysis, we first perform toponym extraction and expansion to obtain a toponym list from news documents. We then propose a matrix factorization method to estimate the location-document relevance scores while simultaneously capturing the correlation of locations and documents. For image enrichment, we propose a method to generate multiple queries from each news document for image search and then employ an intelligent fusion approach to collect a set of images from the search results. Based on the location relevance analysis and image enrichment, we introduce a news browsing system named NewsMap which can support users in reading news via browsing a map and retrieving news with location queries. The news documents with the corresponding enriched images are presented to help users quickly get information. Extensive experiments demonstrate the effectiveness of our approaches.","PeriodicalId":318758,"journal":{"name":"Proceedings of the 19th ACM international conference on Multimedia","volume":"3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117080441","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 29
StoryImaging: a media-rich presentation system for textual stories StoryImaging:文本故事的富媒体呈现系统
Pub Date : 2011-11-28 DOI: 10.1145/2072298.2072451
Genliang Guan, Zhiyong Wang, Xiansheng Hua, D. Feng
In this demo, we develop the StoryImaging system to illustrate a textual story with both images harvested from the Web and synthesized speech. At the backend, a story is firstly processed to identify key terms such as named entities and to obtain the story summary. With the aid of commercial search engines, images are then collected from the Web for those key terms and re-ranked by taking the summary as context. At last, images are clustered to provide an overview of the story. At the web-based frontend, the user interface has been tailored to both improve information comprehension and provide engaging and explorative experiences for users by closely bridging textual and visual modalities.
在这个演示中,我们开发了StoryImaging系统,用从Web获取的图像和合成语音来演示文本故事。在后端,首先对故事进行处理,以识别关键术语(如命名实体)并获得故事摘要。在商业搜索引擎的帮助下,然后从网络上收集这些关键词的图像,并通过将摘要作为上下文重新排序。最后,对图像进行聚类以提供故事的概述。在基于web的前端,用户界面经过了定制,通过紧密连接文本和视觉模式,既提高了信息理解能力,又为用户提供了引人入胜的探索体验。
{"title":"StoryImaging: a media-rich presentation system for textual stories","authors":"Genliang Guan, Zhiyong Wang, Xiansheng Hua, D. Feng","doi":"10.1145/2072298.2072451","DOIUrl":"https://doi.org/10.1145/2072298.2072451","url":null,"abstract":"In this demo, we develop the StoryImaging system to illustrate a textual story with both images harvested from the Web and synthesized speech. At the backend, a story is firstly processed to identify key terms such as named entities and to obtain the story summary. With the aid of commercial search engines, images are then collected from the Web for those key terms and re-ranked by taking the summary as context. At last, images are clustered to provide an overview of the story. At the web-based frontend, the user interface has been tailored to both improve information comprehension and provide engaging and explorative experiences for users by closely bridging textual and visual modalities.","PeriodicalId":318758,"journal":{"name":"Proceedings of the 19th ACM international conference on Multimedia","volume":"65 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117340968","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
期刊
Proceedings of the 19th ACM international conference on Multimedia
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
﹀
mjeditor
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1