Multimodal Multisensor Activity Annotation and Recording Tool<\/h2>\n\n\n\n
Wearable and ubiquitous sensor systems provide physiological data that can be embedded into intelligent user interfaces. Including data should enhance humans and computers such that interaction experience is improved. We provide a tool for recording and one for annotating multimodal data from multiple sensors under an open source license.<\/p>\n\n\n\n
Recording | MMART<\/strong><\/h3>\n\n\n\nThe M<\/strong>ultimodal M<\/strong>ultisensor A<\/strong>ctivity R<\/strong>ecording T<\/strong>ool (MMART) supports common 2D\/3D cameras and interaction devices (Myo, Leap Motion) to be synchronously recorded. You can find the source code and further instructions on Github<\/a>.<\/p>\n\n\n\n![\"Recording](\"\/wp-content\/uploads\/annotool_rec.png\")
<\/figure>\n\n\n\nAnnotating | MMAAT<\/strong><\/h3>\n\n\n\nThe data format is shared with our M<\/strong>ultimodal M<\/strong>ultisensor A<\/strong>ctivity A<\/strong>nnotation T<\/strong>ool (MMAAT) that allows for intuitive labeling of the data providing multiple viewports for visualizing data. The resulting supervised data can be exported for machine learning purposes. You can find the source code and further instructions on Github<\/a>.<\/p>\n\n\n\n![\"Annotation](\"\/wp-content\/uploads\/sepant_ui.png\")
<\/figure>\n\n\n\nCitation<\/strong>
If you use our software please cite our corresponding publication<\/a>.<\/p>\n\n\n\nFine-tuning deep CNN models on specific MS COCO categories<\/h2>\n\n\n\n![\"py-faster-rcnn-ft-demo2\"](\"\/wp-content\/uploads\/py-faster-rcnn-ft-demo2-268x300.png\")
<\/a><\/figure><\/div>\n\n\n\n
Fine-tuning of a deep convolutional neural network (CNN) is often desired. We aim at using fine-tuned image classification models for, e.g., interactive machine learning and multimodal interaction. To this end, we needed an easy-to-use tool for fine-tuning or re-training state-of-the-art deep neural networks on custom datasets or subsets of existing corpora.<\/p>\n\n\n\n
py-faster-rcnn-ft<\/strong>
We forked the original version of py-faster-rcnn<\/a> for adding changes relevant to our research. The py-faster-rcnn-ft<\/strong> software library can be used to f<\/strong>ine-t<\/strong>une the VGG CNN M 1024 model on custom subsets of the Microsoft Common Objects in Context (MS COCO) dataset. For example, we improved the procedure so that the user does not have to look for suitable image files in the dataset by hand which can then be used in the demo program. Our Implementation randomly selects images that contain at least one object of the categories on which the model is fine-tuned.<\/p>\n\n\n\n![\"py-faster-rcnn-ft\"](\"\/wp-content\/uploads\/py-faster-rcnn-ft.png\")
<\/a><\/figure>\n\n\n\npy-faster-rcnn-ft is publicly available on Github<\/a>.<\/p>\n\n\n\nCitation<\/strong><\/p>\n\n\n\nIf you use our software for a research project, we appreciate a reference to our corresponding paper<\/a> (PDF<\/a>).<\/p>\n\n\n\nBibTeX entry:
\n\t@article{Sonntag2017a,
\n\t\ttitle = {{Fine-tuning deep CNN models on specific MS COCO categories}},
\n\t\tauthor = {Sonntag, Daniel and Barz, Michael and Zacharias, Jan and Stauden, Sven and Rahmani, Vahid and F\u00f3thi, \u00c1ron and L\u0151rincz, Andr\u00e1s},
\n\t\tarchivePrefix = {arXiv},
\n\t\tarxivId = {1709.01476},
\n\t\teprint = {1709.01476},
\n\t\tpages = {0--3},
\n\t\turl = {http:\/\/arxiv.org\/abs\/1709.01476},
\n\t\tyear = {2017}
\n\t}<\/code><\/p>\n","protected":false},"excerpt":{"rendered":"GitHub Repository We publish selected software packages via GitHub. You can find our most recent developments on https:\/\/github.com\/DFKI-Interactive-Machine-Learning\/ Multimodal Multisensor Activity Annotation and Recording Tool Wearable and ubiquitous sensor systems provide physiological data that can be embedded into intelligent user … Continue reading
<\/figure>\n\n\n\nAnnotating | MMAAT<\/strong><\/h3>\n\n\n\nThe data format is shared with our M<\/strong>ultimodal M<\/strong>ultisensor A<\/strong>ctivity A<\/strong>nnotation T<\/strong>ool (MMAAT) that allows for intuitive labeling of the data providing multiple viewports for visualizing data. The resulting supervised data can be exported for machine learning purposes. You can find the source code and further instructions on Github<\/a>.<\/p>\n\n\n\n<\/figure>\n\n\n\nCitation<\/strong>
If you use our software please cite our corresponding publication<\/a>.<\/p>\n\n\n\nFine-tuning deep CNN models on specific MS COCO categories<\/h2>\n\n\n\n<\/a><\/figure><\/div>\n\n\n\n
Fine-tuning of a deep convolutional neural network (CNN) is often desired. We aim at using fine-tuned image classification models for, e.g., interactive machine learning and multimodal interaction. To this end, we needed an easy-to-use tool for fine-tuning or re-training state-of-the-art deep neural networks on custom datasets or subsets of existing corpora.<\/p>\n\n\n\n
py-faster-rcnn-ft<\/strong>
We forked the original version of py-faster-rcnn<\/a> for adding changes relevant to our research. The py-faster-rcnn-ft<\/strong> software library can be used to f<\/strong>ine-t<\/strong>une the VGG CNN M 1024 model on custom subsets of the Microsoft Common Objects in Context (MS COCO) dataset. For example, we improved the procedure so that the user does not have to look for suitable image files in the dataset by hand which can then be used in the demo program. Our Implementation randomly selects images that contain at least one object of the categories on which the model is fine-tuned.<\/p>\n\n\n\n<\/a><\/figure>\n\n\n\npy-faster-rcnn-ft is publicly available on Github<\/a>.<\/p>\n\n\n\nCitation<\/strong><\/p>\n\n\n\nIf you use our software for a research project, we appreciate a reference to our corresponding paper<\/a> (PDF<\/a>).<\/p>\n\n\n\nBibTeX entry:
\n\t@article{Sonntag2017a,
\n\t\ttitle = {{Fine-tuning deep CNN models on specific MS COCO categories}},
\n\t\tauthor = {Sonntag, Daniel and Barz, Michael and Zacharias, Jan and Stauden, Sven and Rahmani, Vahid and F\u00f3thi, \u00c1ron and L\u0151rincz, Andr\u00e1s},
\n\t\tarchivePrefix = {arXiv},
\n\t\tarxivId = {1709.01476},
\n\t\teprint = {1709.01476},
\n\t\tpages = {0--3},
\n\t\turl = {http:\/\/arxiv.org\/abs\/1709.01476},
\n\t\tyear = {2017}
\n\t}<\/code><\/p>\n","protected":false},"excerpt":{"rendered":"GitHub Repository We publish selected software packages via GitHub. You can find our most recent developments on https:\/\/github.com\/DFKI-Interactive-Machine-Learning\/ Multimodal Multisensor Activity Annotation and Recording Tool Wearable and ubiquitous sensor systems provide physiological data that can be embedded into intelligent user … Continue reading
<\/figure>\n\n\n\nCitation<\/strong> py-faster-rcnn-ft<\/strong> py-faster-rcnn-ft is publicly available on Github<\/a>.<\/p>\n\n\n\n Citation<\/strong><\/p>\n\n\n\n If you use our software for a research project, we appreciate a reference to our corresponding paper<\/a> (PDF<\/a>).<\/p>\n\n\n\n BibTeX entry: GitHub Repository We publish selected software packages via GitHub. You can find our most recent developments on https:\/\/github.com\/DFKI-Interactive-Machine-Learning\/ Multimodal Multisensor Activity Annotation and Recording Tool Wearable and ubiquitous sensor systems provide physiological data that can be embedded into intelligent user … Continue reading
If you use our software please cite our corresponding publication<\/a>.<\/p>\n\n\n\nFine-tuning deep CNN models on specific MS COCO categories<\/h2>\n\n\n\n
<\/a><\/figure><\/div>\n\n\n\n
Fine-tuning of a deep convolutional neural network (CNN) is often desired. We aim at using fine-tuned image classification models for, e.g., interactive machine learning and multimodal interaction. To this end, we needed an easy-to-use tool for fine-tuning or re-training state-of-the-art deep neural networks on custom datasets or subsets of existing corpora.<\/p>\n\n\n\n
We forked the original version of py-faster-rcnn<\/a> for adding changes relevant to our research. The py-faster-rcnn-ft<\/strong> software library can be used to f<\/strong>ine-t<\/strong>une the VGG CNN M 1024 model on custom subsets of the Microsoft Common Objects in Context (MS COCO) dataset. For example, we improved the procedure so that the user does not have to look for suitable image files in the dataset by hand which can then be used in the demo program. Our Implementation randomly selects images that contain at least one object of the categories on which the model is fine-tuned.<\/p>\n\n\n\n<\/a><\/figure>\n\n\n\n
\n\t@article{Sonntag2017a,
\n\t\ttitle = {{Fine-tuning deep CNN models on specific MS COCO categories}},
\n\t\tauthor = {Sonntag, Daniel and Barz, Michael and Zacharias, Jan and Stauden, Sven and Rahmani, Vahid and F\u00f3thi, \u00c1ron and L\u0151rincz, Andr\u00e1s},
\n\t\tarchivePrefix = {arXiv},
\n\t\tarxivId = {1709.01476},
\n\t\teprint = {1709.01476},
\n\t\tpages = {0--3},
\n\t\turl = {http:\/\/arxiv.org\/abs\/1709.01476},
\n\t\tyear = {2017}
\n\t}<\/code><\/p>\n","protected":false},"excerpt":{"rendered":"