41 results about "Visual modality" patented technology

Methods and systems for facilitating the selecting and / or the purchasing of items are provided. Items to be purchased may be clothing items. The use of a visualization pane comprising an avatar may be used to facilitate item selection. The avatar may represent a person such as a user and clothing items may be represented on the avatar so as to provide a preview of how the clothing items would look on the user. Searching for purchasable items may be done textually or visually using key images or by a combination of both. Key images selected from a dictionary of key images may to represent search criteria. Coupon offers may be presented to a user which may be redeemable instantly or later at a store location.

The present invention relates to a method for assessing a learner's pronunciation through voices and images, in which variation of the teacher's and the learner's lips are compared in a visual manner. Accordingly, incorrect pronunciation can be pointed out and recorded for assessment and rectification.

A multi-modal pre-training method based on image-text linear combination belongs to the technical field of image-text multi-modal retrieval, and comprises the following steps: S1, respectively carrying out feature extraction on a text and an image; s2, establishing a relation between two modes of a text and an image in an interaction layer; s2.1, jointly inputting the feature vectors of the visual modality and the language modality obtained in the step S1 into an interaction layer of a multi-modality pre-training model; s2.2, utilizing an attention mechanism in the Transform to enable the two modes to be connected with each other; s3, taking the image-text matching or shielding language model as a pre-training target, and training the model to be available; and S4, taking a specific application scene and a downstream task as training targets, carrying out fine tuning training on the pre-training model, and enabling the performance of the model to be optimal in the scene. The training method provided by the invention solves the bottleneck problem of model operation time and the performance problem of the improved pre-training model after fine tuning, and has relatively important scientific significance and practical application value.

The present invention extends to methods, systems, and computer program products for automatically visually presenting information relevant to an utterance. Natural language expressions from conversation participants are received and processed to determine a topic and concepts, a search finds relevant information and it is visually displayed to an assisted user. Applications can include video conferencing, wearable devices, augmented reality, and heads-up vehicle displays. Topics, concepts, and information search results are analyzed for relevance and non-repetition. Relevance can depend on a user profile, conversation history, and environmental information. Further information can be requested through non-verbal modes. Searched and displayed information can be in languages other than that spoken in the conversation. Many-party conversations can be processed.

A method and system for use in a user system, for accessing information related to a physical document. An electronic copy of an existing physical document is identified and located. The electronic copy of the physical document is an exact replica of the physical document. One or more pages of the physical document are identified. A selected part of the physical document is identified using the position of points on the identified one or more pages of the physical document and in response, data related to the selected part of the physical document is retrieved from the electronic copy of the physical document. The retrieved data is presented visually to a visually impaired person or orally to a blind person on the user system, which enables the visually impaired person to see or hear, respectively, the retrieved data.

The present invention claims and discloses a color and, or symbol coded display on a digital interactive badge for communicating a permission to approach and activate further digital content interaction. A non-verbal person to person ‘line of sight’ digital or electronic communication protocol standard (NVP) which allows one person to communicate with another using a direct line of sight signal from one interactive badge or screen to another. It allows the user to communicate and display their feelings, emotions, states of mind and consciousness, their like-mindedness with others, their social media activity, images, favorite brands, videos, purchased items and advertising all inside a new and comprehensive language NVP and in a visual way. NVP also allows the user to send invites to approach, talk, interact and to send digital information to their NVP capable interactive badge or screen or static receiver. NVP has been designed to run on hardware that can be programmed from the NVP interface and is designed specifically for the wearable technology market.

The invention discloses a visual positioning system, battery replacement equipment and a battery replacement control method. The visual positioning system comprises a first visual sensor, a second visual sensor and a position acquisition unit, wherein the first visual sensor is used for acquiring a first image of a first position of the target device; the second visual sensor is used for acquiring a second image of a second position of the target device; the position acquisition unit is used for acquiring position information of the target device according to the first image and the second image. High positioning precision is obtained through a visual mode, and accurate positioning between the battery replacement equipment and the vehicle to be subjected to battery replacement is realized.

Disclosed herein are systems and methods for converting audio-video content into audio-only content. Audio-video content is readily accessible, but for various reasons users often cannot consume content visually. In those circumstances, for example, when a user is interrupted during a movie to drive to pick up a spouse or child, the user may not want to forego consuming the audio-video content. The audio-video content can be converted into audio only content for the user to aurally consume, allowing the user to consume the content despite interruptions or other reasons for which the audio-video content cannot be consumed visually.

The present invention proposes an adversarial double-contrast self-supervised learning method for cross-modal lip reading, which includes a visual encoder, an audio encoder, two multi-scale temporal convolutional networks with average pooling, an identity discriminator, and a model state classifier. The method learns effective visual representations by combining audio-visual synchrony-based dual contrastive learning, identity adversarial training, and modality adversarial training. In double-contrastive learning, the noise-contrastive estimate is used as the training objective to distinguish real samples from noise samples. In the adversarial training, an identity discriminator and a modality classifier are proposed for audio-visual representation, the identity discriminator is used to distinguish whether the input visual features have a common identity, and the modality classifier is to predict whether the input features belong to the visual modality or not. The audio modality is then utilized for adversarial training using a momentum gradient reversal layer.

A voice detection method, a training method of a prediction model, a device, equipment and a medium, belonging to the technical field of voice interaction. A multi-modal speech end point detection method, which uses the model to recognize the captured face image, thereby predicting whether the user has the intention to continue speaking, combined with the prediction results, to determine whether the collected audio signal is the speech end point , because on the basis of the acoustic features, the features of the visual modality of the face image are also integrated for detection, even when the background noise is strong or the user pauses during speech, the face image can be used to accurately judge Whether the voice signal is the end point of the voice, thus avoiding the interference caused by background noise and speech pauses, thereby avoiding the problem of too late or premature detection of the end of the voice interaction caused by the interference of background noise and speech pauses, and improving the Accuracy in detecting speech end points.

The invention relates to a lubricating oil film level tester and a test method thereof. The tester includes a capacitance sensor arranged in an oil tank and a measurement module connected to the capacitance sensor. The measurement module includes a processor configured to is: Obtain the first measured value ε for the first medium representing the lowest oil film level min , to obtain a second measurement ε for the second medium representing the highest oil film level max , to obtain the third measured value ε of the third medium measured by the capacitive sensor x , calculate the oil film level of the third medium and feedback it to the user through visual and / or auditory means, where the first measured value ε min , the second measured value ε max and the third measured value ε x They are proportional to the first dielectric coefficient of the first medium, the second dielectric coefficient of the second medium, and the third dielectric coefficient of the third medium, and the proportional ratios are the same. The third medium is unused lubricating Oil.

The invention discloses an image tagging method and system based on a typical correlation model of weak matching probability. The invention relates to the technical field of network cross-media information processing, including acquiring tagged images and unlabeled images in an image database, and extracting the tagged images respectively. With the image features and text features of the unlabeled image, a matched sample set and an unmatched sample set are generated, the matched sample set includes an annotated image feature set and an annotated text feature set, and the unmatched sample set includes An unlabeled image feature set and an unlabeled text feature set; according to the matched sample set and the unmatched sample set, train the weak matching probability typical correlation model; through the weak matching probability typical correlation model, treat Annotate images for annotation. The present invention simultaneously uses marked images and their keywords and unmarked images to learn the association between the visual modality and the text modality, and accurately marks unknown images.

A system, method and computer program produce for handling and minimizing miscommunication and transformation of tokens that are processed by humans, either verbally or in writing, during some part of a usage scenario. This is accomplished by filtering out confusing tokens, as determined by calculating a distance metric for each token. A distance metric may be calculated along a print modality, a visual modality or a verbal modality.

The invention relates to a visual detection method and device for riverway drifts. The method comprises the following steps: acquiring a riverway image; extracting a region of interest of the riverway image based on a preset extraction method; carrying out difference feature extraction on the region of interest based on a twin neural network; and identifying difference features extracted by the twin neural network based on an SSD target detection algorithm to determine the type and position of a drift object. The method has the beneficial effects that from the perspective of computer vision, a target detection model based on the twin network is constructed, appearance of river surface drifts is regarded as feature differences, detection of the drifts is converted into identification and positioning of difference features, and thus category judgment, position positioning and area and flow velocity calculation of the drifts are accurately realized. The river drift detection is realized in a visual mode, the labor cost is low, the dependence on hardware equipment is weak, the maintenance is convenient, and the detection precision is high.

The invention provides a multi-modality vocabulary representing method. The multi-modality vocabulary representing method comprises the steps of calculating a text representing vector of a vocabulary to be represented in text modality and a picture representing vector of the vocabulary to be represented in visual modality; inputting the text representing vector into a pre-established text modality weight model to obtain the weight of the text representing vector in the text modality; inputting the picture representing vector into a pre-established visual modality weight model to obtain the weight of the picture representing vector in picture modality; conducting calculation to obtain a multi-modality vocabulary representing vector according to the text representing vector, the picture representing vector and weights corresponding to the text representing vector and the picture representing vector respectively. The text modality weight model is a neural network model of which input is the text representing vector and output is the weight of the text representing vector in the corresponding text modality; the visual modality weight model is a neural network model of which input is the picture representing vector and output is the weight of the picture representing vector in the corresponding visual modality.

The present invention is to provide a terminal for performing a secretarial function for assisting a person's memory at home or in a company, and a system comprising the same. The terminal comprises: a camera module configured to capture an image of a predetermined space when a person's motion in the predetermined space is detected; and a memory for storing the image captured by the camera module, wherein when a user request is received, the position of an object corresponding to the user request is detected using the image stored in the memory, and guide information can be outputted in at least one manner of a visual manner, an auditory manner and a tactile manner so that the detected position is guided.

The present invention proposes a device and method for clock calibration. The current clock time of the clock is photographed in real time by using the camera module in a visual way. The image processing unit receives the current clock time image and processes the image data according to an algorithm to obtain the adjusted clock time. , obtain the adjusted clock time, align the origin of the clock calibration device with the clock origin, absorb the clock hands, and adjust the clock time according to the obtained adjusted clock time. The present invention utilizes the cyclic data processing mode of the image processing unit to obtain the current clock of the watch for precise calibration, which is more convenient, more efficient and more intelligent.

A wearable visual communications unit for automatically presenting information visually to third parties, the visual communication unit including a visual display, a microprocessor, a memory unit, a location tracker module in communication with the microprocessor, and a command signal generating unit in communication with the visual display, where the location tracker module is configured to track location of the wearable visual communication unit and is configured to generate location based data, where the micro-processor is operable to receive the location based data and to execute code stored in the memory unit to select a required visual presentation, and where the command signal generating unit is configured to generate at least one command signal to control the visual display to present the required visual presentation.

The invention relates to a button fastening machine with a function of automatic button adjustment, which comprises a machine head, a control system, an automatic button feeding device, a button fixing structure, an image collecting device, and a button adjusting mechanism. The image collecting device is used for collecting surface images of buttons on the button fixing structure. The control system determines the actual direction of the button by comparing the collected surface image information with the pre-stored button image data and sends the button adjustment instruction to the button adjusting mechanism that adjusts the button direction according to the button adjustment instruction. The button fixing structure swings between the connecting position, the identifying position and thefastening position through a swing rod. The button fastening machine identifies the button angle and the front and the back with a visual presentation, then adjusts the button according to the comparison result of the actual direction of the button and the ideal direction of the button. By automatically obtaining the button which meets the requirement of the button direction, the labor is greatlysaved and the working efficiency of the button fastening machine is improved.

A controller for real-time visual display of music includes a music analysis module and a display control module. The music analysis module receives an audio input, determines human perceived musical structures, human felt affect and emotion as a function of the audio input, and outputs a signal corresponding to the determined structure, affect and emotion. The display control module is operatively coupled to the music analysis module and receives the signal and controls a visual display as a function thereof to express the determined musical structure, affect and emotion in a visual manner.