I am the inventor or co-inventor of 11 patents covering 5 different technological issues (for more details and full list of patents, you can visit the European Patent Office web site):

Intrinsically motivated reinforcement learning, life-long learning, autonomous development of intelligence in self-developping devices

Together with Frederic Kaplan, I am the co-inventor of 2 patents which describe a machine learning technology related to the capability of a learning machine to select autonomously its training examples and self-develop repertoires of skill through life-long learning. A part of this technology has been integrated in the Sony Qrio humanoid robot:

  1. Automated Action-Selection System and Method , and Application Thereof to Training Prediction Machines and Driving the Development of Self-Developing Devices (27 july 2004)
    Abstract: In order to promote efficient learning of relationships inherent in a system or setup S described by system-state and context parameters, the next action to take, affecting the setup, is determined based on the knowledge gain expected to result from this action. Knowledge-gain is assessed “locally” by comparing the value of a knowledge-indicator parameter after the action with the value of this indicator on one or more previous occasions when the system-state/context parameter(s) and action variable(s)=had similar values to the current ones. Preferably the “level of knowledge” is assessed based on the accuracy of predictions made by a prediction module. This technique can be applied to train a prediction machine by causing it to participate in the selection of a sequence of actions. This technique can also be applied for managing development of a self-developing device or system, the self-developing device or system performing a sequence of actions selected according to the action-selection technique.
  2. Architecture for self-developing devices (6th june 2003)
    Abstract: The present invention relates to an architecture for self-developing devices. More particularly, the invention relates to self-developing devices adapted so as to be capable of continuously developing new know-how (this is sometimes referred to as the capacity to engage in “lifelong learning”). A self-developing device (1) capable of open-ended development makes use of a special motivational system for selecting which action should be taken on the environment by an associated sensory-motor apparatus (2). For a given candidate action, a motivational module (11) calculates a reward associated with the corresponding values that would be taken by one or more motivational variables that are independent of the nature of the associated sensory-motor apparatus. Preferred motivational variables are dependent on the developmental history of the device (1), and include variables quantifying the predictability, familiarity and stability of sensory-motor variables serving as the inputs to the device (1). The sensory-motor variables represent the status of the external environment and/or the internal resources (3) of the sensory-motor apparatus (2) whose behaviour is controlled by the self-developing device (1). Open-ended development is enabled by attributing a reward which is proportional to the rate of change of the history-dependent motivational variables.

Emotional speech synthesis and recognition

I am the inventor of several worldwide patents describing various aspects of technologies for synthesis and recognition of emotional speech with/by computers. This technology has been integrated into the Qrio robot, and into Sony Playstation video games.

  1. Method and apparatus for synthesizing an emotion conveyed on a sound, 13th july 2001
    An emotion conveyed on a sound is synthesised by selectively modifying elementary sound portions (S) thereof prior to delivering the sound through an operator application step (S10, S16, S20) in which at least one operator (OP, OD; OI) is selectively applied to the elementary sound portions (S) to impose a specific modification in a characteristic, such as pitch and/or duration in accordance with an emotion to be synthesised.

    This can be achieved by applying at least one operator (OIrs, OIfs, OIsu, OIsd) to modify an intensity characteristic of an elementary sound portion, and/or an operator (OPrs, OPfs) for selectively causing the time evolution of the pitch of an elementary sound portion (S) to rise or fall according to an imposed slope characteristic (Prs, Pfs), and/or an operator (OPsu, OPsd) for selectively causing the time evolution of the pitch of an elementary sound portion (S) to rise or fall uniformly by a determined value (Psu, Psd), and/or an operator (ODd, ODc) for selectively causing the duration (t1) of an elementary sound portion (S) to increase or decrease by a determined value (D).
  2. Method and apparatus for controlling the operation of an emotion synthesizing device, 14th august 2001
    Abstract: The invention controls the operation of an emotion synthesising device (12), notably of the type where the emotion is conveyed on a sound, having at least one input parameter (Pi) whose value (Ei) is used to set a type of emotion to be conveyed, by making at least one parameter a variable parameter (VPi) over a determined control range, thereby to confer a variability in an amount of the type of emotion to be conveyed. The variable parameter (VPi) can be made variable according to a variation model over the control range, the model relating a quantity of emotion control variable (δ) to the variable parameter, whereby said control variable is used to variably establish a value of said variable parameter.
    Preferably the variation obeys a linear model, the variable parameter being made to vary linearly with a variation in a quantity of emotion control variable (δ).
  3. Emotion recognition method and device, 13th july 2001
    Abstract: Emotion recognition is performed by extracting a set comprising at least one feature derived from a signal, and processing the set of extracted feature(s) to detect an emotion therefrom. The voice signal is low pass filtered prior to extracting therefrom at least one feature of the set. The cut-off frequency for the low pass filtering is typically centered around 250 Hz. The features are e.g. statistical quantities extracted from sampling a signal of the intensity or pitch of the voice signal.
  4. Method and apparatus for voice synthesis and robot apparatus, 11th may 2001
    Abstract: A robot apparatus (1) is capable of audibly expressing an emotion in a manner similar to that performed by a living animal. The robot apparatus (1) utters a sentence by means of voice synthesis by performing a process including the steps of: an emotional state discrimination step (S1) for discriminating an emotional state of an emotion model (73); a sentence output step (S2) for outputting a sentence representing a content to be uttered in the form of a voice; a parameter control step (S3) for controlling a parameter for use in voice synthesis, depending upon the emotional state discriminated in the emotional state discrimination step (S1); and a voice synthesis step (S4) for inputting, to a voice synthesis unit, the sentence output in the sentence output step (S2) and synthesizing a voice in accordance with the controlled parameter.
  5. Sound processing method and device for modifying a sound characteristic, such as an impression of age associated to a voice, 14th august 2001
    Abstract: The invention allows to controllably modify a value of a characteristic, such as an age, associated to a sound contained in sound data (30) such as voice to which is associated a sampling frequency, the sound exhibiting a given value (A) of the characteristic for a first sampling frequency F1. This is achieved by :

    • producing from the sound data at least one sound data portion (UD) containing a sound portion to be reproduced with a modified value of the characteristic, the sound data portion having a first sampling duration R1 at the first sampling frequency F1,
    • establishing (36, 44-48) a second frequency F2 selected as a function of a desired modification in the value of the characteristic,
    • time adjusting (38) the sound data portion (UD) by elaborating a time-adjusted form (UDta) thereof on the basis of a sampling frequency equal to the second frequency F2, the time-adjusting step at least partially tending to cause the sampling duration of the time-adjusted form of sound data portion at the second frequency F2 to match the first sampling duration R1, and
    • playing the time adjusted sound data portion (UDta).
    Preferably, the time-adjusting step is performed by means of a Pitch-Synchronous Overlap and Add (PSOLA) algorithm (38) or the like.
  6. Method and apparatus for speech synthesis program, recording medium, method and apparatus for generating constraint information and robot apparatus, 15 march 2002 (together with  Makoto Akabane, Erika Kobayashi, Kenichiro Kobayashi, Toshiyuki Kumakura, Tomoaki Nitta, Nobuhide Yamazaki)
    Abstract: The emotion is to be added to the synthesized speech as the prosodic feature of the language is maintained. In a speech synthesis device 200, a language processor 201 generates a string of pronunciation marks from the text, and a prosodic data generating unit 202 creates prosodic data, expressing the time duration, pitch, sound volume or the like parameters of phonemes, based on the string of pronunciation marks. A constraint information generating unit 203 is fed with the prosodic data and with the string of pronunciation marks to generate the constraint information which limits the changes in the parameters to add the so generated constraint information to the prosodic data. A emotion filter 204, fed with the prosodic data, to which has been added the constraint information, changes the parameters of the prosodic data, within the constraint, responsive to the feeling state information, imparted to it. A waveform generating unit 205 synthesizes the speech waveform based on the prosodic data the parameters of which have been changed.

Robot behavioural shaping through clicker-training

Together with Frederic Kaplan, I am the co-inventor of a technique inspired from clicker -training which allows to teach various behaviours to a robot in a way which is quite similar to way real dogs or dolphins are trained. This technology has been integrated in the software of the commercial version of the Sony AIBO robot.

  1. Training of autonomous robots, 30th april 2001
    Abstract: A clicker-training technique developed for animal training is adapted for training robots, notably autonomous animal-like robots. In this robot-training method, a behaviour (for example, (DIG)) is broken down into smaller achievable responses ((SIT)-(HELLO)-(DIG)) that will eventually lead to the desired final behaviour. The robot is guided progressively to the correct behaviour through the use, normally the repeated use, of a secondary reinforcer. When the correct behaviour has been achieved, a primary reinforcer is applied so that the desired behaviour can be “captured”. This method can be used for training a robot to perform, on command, rare behaviours or a sequence of behaviours (typically actions). This method can also be used to ensure that a robot is focusing its attention upon a desired object.

Touchpad based human-robot interface for realizing joint attention

Together with Pierre Rouanet, we co-invented a human-robot interface which allows to efficiently show novel unknown objects to a robot and ensure joint attention. A typical realization of this technology uses smartphones such as the Iphone as a mediator interface.

  1. Device and method for interacting with autonomous robot (25th january 2006)
    Abstract: A novel device and a related method are provided for use with a trigger-and-respond autonomous robot. The device functions both as an electronic repository of graphical images, each containing an encoded instruction, and a presentation mechanism of the encoded instructions to the autonomous robot. The device contains the following major components: an output means both as a display to the user and a display of graphical images to the autonomous robot; an input means via which the user can perform various point-and-select tasks; a non-volatile information repository for storing graphical images; and an organization means which organizes and presents the graphical images in a tree-like or hierarchical manner for efficient search and retrieval.