The Future of Camera Networks: Staying Smart in a Chaotic World

Camera networks become smart when they can interpret video data on board, in order to carry out tasks as a collective, such as target tracking and (re-)identification of objects of interest. Unlike today's deployments, which are mainly restricted to lab settings and highly controlled high-value applications, future smart camera networks will be messy and unpredictable. They will operate on a vast scale, drawing on mobile resources connected in networks structured in complex and changing ways. They will comprise heterogeneous and decentralised aggregations of visual sensors, which will come together in temporary alliances, in unforeseen and rapidly unfolding scenarios. The potential to include and harness citizen-contributed mobile streaming, body-worn video, and robot-mounted cameras, alongside more traditional fixed or PTZ cameras, and supported by other non-visual sensors, leads to a number of difficult and important challenges. In this position paper, we discuss a variety of potential uses for such complex smart camera networks, and some of the challenges that arise when staying smart in the presence of such complexity. We present a general discussion on the challenges of heterogeneity, coordination, self-reconfigurability, mobility, and collaboration in camera networks.

[1]  Amit K. Roy-Chowdhury,et al.  Distributed Multi-Target Tracking and Data Association in Vision Networks , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Bernhard Rinner,et al.  Dynamic Reconfiguration in Camera Networks: A Short Survey , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[3]  Peter R. Lewis,et al.  Online Multi-object k-coverage with Mobile Smart Cameras , 2017, ICDSC.

[4]  Kay Römer,et al.  The design space of wireless sensor networks , 2004, IEEE Wireless Communications.

[5]  Marco Platzner,et al.  Self-aware Computing: Introduction and Motivation , 2016, Self-aware Computing Systems.

[6]  Shaogang Gong,et al.  Tracking multiple people with a multi-camera system , 2001, Proceedings 2001 IEEE Workshop on Multi-Object Tracking.

[7]  Shai Avidan,et al.  Ensemble Tracking , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Lynne E. Parker,et al.  Cooperative multi-robot observation of multiple moving targets , 1997, Proceedings of International Conference on Robotics and Automation.

[9]  Laurent Itti,et al.  Distributed biologically-based real-time tracking in the absence of prior target information , 2005, SPIE Optics East.

[10]  Stephan Winter,et al.  A Spatio-Temporal Model Towards Ad-Hoc Collaborative Decision-Making , 2010, AGILE Conf..

[11]  Anh Vu,et al.  Feature mapping and state estimation for highly automated vehicles , 2015 .

[12]  W. Li,et al.  Sensor selection for improving accuracy of target localisation in wireless visual sensor networks , 2012, IET Wirel. Sens. Syst..

[13]  Gian Luca Foresti,et al.  Distributed and Unsupervised Cost-Driven Person Re-Identification , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[14]  Bernhard Rinner,et al.  Resource-Aware Coverage and Task Assignment in Visual Sensor Networks , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[15]  Marco Platzner,et al.  Self-aware Computing Systems , 2016, Natural Computing Series.

[16]  Bernhard Rinner,et al.  An Autonomous Multi-UAV System for Search and Rescue , 2015, DroNet@MobiSys.

[17]  Amit K. Roy-Chowdhury,et al.  Network Consistent Data Association , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Peter R. Lewis,et al.  It's Good to Be Different: Diversity, Heterogeneity, and Dynamics in Collective Systems , 2014, 2014 IEEE Eighth International Conference on Self-Adaptive and Self-Organizing Systems Workshops.

[19]  Annie S. Wu,et al.  On the Impact of Variation on Self-Organizing Systems , 2011, 2011 IEEE Fifth International Conference on Self-Adaptive and Self-Organizing Systems.

[20]  Yiming Li,et al.  Utility-Based Camera Assignment in a Video Network: A Game Theoretic Framework , 2011, IEEE Sensors Journal.

[21]  Jindong Tan,et al.  Selection and Navigation of Mobile Sensor Nodes Using a Sensor Network , 2005, Third IEEE International Conference on Pervasive Computing and Communications.

[22]  Benoit Baudry,et al.  The Multiple Facets of Software Diversity , 2014, ACM Comput. Surv..

[23]  João Pedro Hespanha,et al.  Cooperative Search by UAV Teams: A Model Predictive Approach using Dynamic Graphs , 2011, IEEE Transactions on Aerospace and Electronic Systems.

[24]  Andrea Cavallaro,et al.  Self-Reconfigurable Smart Camera Networks , 2014, Computer.

[25]  Bernhard Rinner,et al.  Self-Organising Zooms for Decentralised Redundancy Management in Visual Sensor Networks , 2015, 2015 IEEE 9th International Conference on Self-Adaptive and Self-Organizing Systems.

[26]  Bernhard Rinner,et al.  Static, Dynamic, and Adaptive Heterogeneity in Distributed Smart Camera Networks , 2015, ACM Trans. Auton. Adapt. Syst..

[27]  Bernhard Rinner,et al.  An Introduction to Distributed Smart Cameras , 2008, Proceedings of the IEEE.

[28]  Michael Sonnenschein,et al.  On the Influence of Inter-Agent Variation on Multi-Agent Algorithms Solving a Dynamic Task Allocation Problem under Uncertainty , 2012, 2012 IEEE Sixth International Conference on Self-Adaptive and Self-Organizing Systems.

[29]  Andrea Cavallaro,et al.  Cost-Aware Coalitions for Collaborative Tracking in Resource-Constrained Camera Networks , 2015, IEEE Sensors Journal.

[30]  UĞUR MURAT ERDEM,et al.  Event prediction in a hybrid camera network , 2012, TOSN.

[31]  Tim J. Ellis,et al.  Multi camera image tracking , 2006, Image Vis. Comput..

[32]  Gian Luca Foresti,et al.  Person Reidentification in a Distributed Camera Network Framework , 2017, IEEE Transactions on Cybernetics.

[33]  Vijay Kumar,et al.  Robot and sensor networks for first responders , 2004, IEEE Pervasive Computing.

[34]  Andreas Angerer,et al.  Towards Self-Organizing Swarms of Reconfigurable Self-Aware Robots , 2016, 2016 IEEE 1st International Workshops on Foundations and Applications of Self* Systems (FAS*W).

[35]  Juan A. Rodríguez-Aguilar,et al.  Self-Configuring Sensors for Uncharted Environments , 2010, 2010 Fourth IEEE International Conference on Self-Adaptive and Self-Organizing Systems.

[36]  Ali A. Minai,et al.  Self-Organization of Sensor Networks with Heterogeneous Connectivity , 2010 .

[37]  Scott Gruber,et al.  Flexible multi-agent algorithm for distributed decision making , 2015, 2015 International Conference on Unmanned Aircraft Systems (ICUAS).

[38]  Ada Diaconescu,et al.  The Notion of Self-aware Computing , 2017, Self-Aware Computing Systems.

[39]  Eelke van Foeken,et al.  Adaptive team formation for shared situation awareness , 2012, 2012 15th International Conference on Information Fusion.

[40]  Bernhard Rinner,et al.  Self-aware Object Tracking in Multi-Camera Networks , 2016, Self-aware Computing Systems.

[41]  Demetri Terzopoulos,et al.  Multi-camera Control through Constraint Satisfaction for Persistent Surveillance , 2008, 2008 IEEE Fifth International Conference on Advanced Video and Signal Based Surveillance.

[42]  Hairong Qi,et al.  Achieving k-Barrier Coverage in Hybrid Directional Sensor Networks , 2014, IEEE Transactions on Mobile Computing.

[43]  Stefano Carpin,et al.  Cooperative Observation of Multiple Moving Targets: an algorithm and its formalization , 2007, Int. J. Robotics Res..

[44]  Bernhard Rinner,et al.  Self-Aware and Self-Expressive Camera Networks , 2015, Computer.

[45]  J. Karl Hedrick,et al.  Autonomous UAV path planning and estimation , 2009, IEEE Robotics & Automation Magazine.

[46]  Geoffrey A. Hollinger,et al.  Efficient Multi-robot Search for a Moving Target , 2009, Int. J. Robotics Res..

[47]  Hamid K. Aghajan,et al.  Camera selection for tracking in distributed smart camera networks , 2014, TOSN.

[48]  Xin Yao,et al.  A socio-economic approach to online vision graph generation and handover in distributed smart camera networks , 2011, 2011 Fifth ACM/IEEE International Conference on Distributed Smart Cameras.

[49]  Vijay Kumar,et al.  Distributed Search and Rescue with Robot and Sensor Teams , 2003, FSR.

[50]  Xin Yao,et al.  Socio-economic vision graph generation and handover in distributed smart camera networks , 2014, TOSN.

[51]  Peter R. Lewis.  Self-aware computing systems: From psychology to engineering , 2017, Design, Automation & Test in Europe Conference & Exhibition (DATE), 2017.

[52]  Bernhard Rinner,et al.  Multiscale observation of multiple moving targets using Micro Aerial Vehicles , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[53]  Suresh Singh,et al.  Exploiting heterogeneity in sensor networks , 2005, Proceedings IEEE 24th Annual Joint Conference of the IEEE Computer and Communications Societies..