the system in fig. 1. As used here, a rigid structure is a trace in which the movement also supports the structure, and the position of the trace from the other side is negligible relative. In addition, the semi-rigid structure is one of them that allows the trace of themovement in the same supporting structure, but the movement is greatly limited within a predetermined range. It may be based on several factors, such as the type of material, the strength of the supporting structure and the application of the supporting structure, and the amount of trademarks between movements. For example, a flexible body adaptation, depending on the materials used and the construction method, may qualify as a "rigid" or "semi-rigid" supporting structure, which shows the appropriate level of rigidity in thedisclosed technology. In addition, bands attached to a suit can also qualify as rigid or semi-rigid support structures.
in some embodiments, the spacing of the trademark signs on the supporting structure may be determined by the technology discussed in detail below, so that the spacing does not need to know a priori. The system can use the trackdifferent logo of the supporting structure of one or more cameras. These markers can be used to estimate the motion of the supporting structure (for example, through the position and direction of time in three-dimensional space). The knowledge that each supported structure is rigid (or semi-rigid) may beused in the estimation process discussed below, and may help to reconstruct the motion from a single camera.
in some implementations, it marks a high contrast material, and you can also choose to light up the light-emitting diode (LED) luminescent material and see it in the dark. These lighting qualities can make the trademark on the camera capturethe object remarkable, in poor light or dark conditions. For example, an actor is being filmed walking from a well-lit area to a shadow area. Movement to the shadow area, although the actor's logo may be captured because themarks glow or glow.
During the operation, there may be a motion sequence recorded by one or more cameras, and 3d scanner images may be stored for processing, such as magnetic tape or disk storage devices. At the same time, the 3d scanner image can be directly transmitted to the processing equipment for processing inreal time or near real time.
The operator or algorithm calculates the position of the camera and its visual characteristics, lens distortion and direction, such as the field of view of the camera, while the sequence is recorded. For example, an algorithm can get enough marks and traces of information, such as numbers, recognition and location, and the camera's position related to the traces taken by the camera.
fig. 2 shows a flow chart illustrating an example of a method of processing motion capture data frames to calculate the relationship between structural markers of a tracked base object. In Embodiment 2 as shown in the figure, the supporting structure is a virtual structure, such as the object that generates a virtual skeleton structure model, in step 22. For example, the support structure may be a band that surrounds an actor's arm. Bandcan is annular and occupies the definition of x, y and z axes in three-dimensional space. The point (for example, X = Y = Z = ) in the object space of the values of the x, y and z axes on the ring is the geometric center of the band. In some embodiments, the geometriccenter may be largely aligned with the geometric center of a virtual arm bone. In other embodiments, the virtual arm bone with the geometric center may cancel the geometric center of the phase ring.
The camera captures, or records, the mark where the supporting structure and recording are used to identify the mark, as shown in step 24. An algorithm can calculate one or more light traces, from one or morecameras through one or more records to mark the extension, as shown in step 26. For example, the algorithm can calculate the traces of two lights. These two kinds of ray traces may extend the first recording mark and the second recording mark, respectively, from a single camera view.
the possible supporting structure, the distance between the marks on this structure, the rigidity of the structure, and the geometric structure of the information system (discussed in more detail below) or auser input. This information may define that traces of a configuration description structure should or may occur. For the convenience of explaining the configuration information, it is described here as a virtual supporting structure in three-dimensional space.
An algorithm can estimate the 3D ray tracing of the current virtual bone structure in the 3D direction aligned with the virtual supporting structure, as shown in step 21. Several different types of ofsolving algorithms may be used to achieve this adjustment, such as maximum likelihood estimation or nonlinear minimization of an error function using a heuristic of Levenberg marquardt. The solution of the algorithm is to move the three-dimensional position of the trace of the virtual support structure and lie on the 3Dray trace. After alignment, the current direction of the virtual bone structure is known, because the virtual support structure has a significantly fixed relationship with its corresponding direction of the virtual bone structure.
in some embodiments, two or more cameras can record multiple observations of the same trademark. The alignment algorithm may use additional records of each mark as a location to solve the additional constraints of calculation. If there is no trademark Ona support structure captured by a camera, observing the signs on other support structures can be used to estimate the position of the uncaught support structure, or at least limit its space area. For example, the supporting structurearound and the actor's elbow may not be visible, but the supporting structure around an actor's wrist may be visible. The position of the wrist supporting structure corresponding to a virtual wrist bone may be calculated in the manner described above. Because the position of virtual wrist bone is well known, the position of a virtual elbow bone corresponding to the elbow support structure may be limited to a limited area. This constraint can be based on the motion model of the object, which can fundamentally define the physical structure of the actor, including the length of the actor's bones connecting the elbow to the wrist. The motion model of the object is discussed in detail below.
In view of these estimated ranges or the three-dimensional position of the virtual bone structure, the motion of the basic object can be predicted. In one of the motions in which a position of a mark can not be used for estimation (such as some parts are notobserved by any camera), the physical properties of one or more objects, such as the natural limitation of an actor's leg within the range of motion, can be used to infer the most possible position of the trademark (so the movement of the object) based on the virtual bone structure and the known three-dimensional position and estimation, and the currentobservation obtained from the employment of the virtual bone structure before and in the future.
optionally, the motion model of an object can be accessed, as shown in step 28, to further restrict the known configuration and ray tracing algorithms for solving the alignment marks. The motion model of an object can substantiallydefine the motion of the underlying object. The mass of an object and the properties of the joint, the motion amplitude, the speed and the acceleration of a specific person's physique can be deduced based on the previously captured motion information theobject. For example, several cameras and traditional motion capture devices capture and record the actor's movements and the position where the triangle is placed on the actor's white ball, the length of an actor's bones and the typical range of motion. Thecaptured motion may be used to derive a model of actors.
after the captured 3d scanner image data frame is processed, the method of 2 can judge whether there are more frames to process the data, as shown in step 212. If there are more frames, the method can return to step 22. Otherwise, the method mayend.
fig. 1 shows a diagram illustrating an example of a motion capture system 1. As shown in the figure. For example, a 1 system includes a support structure number 11 and a symbol 112 attached to each support structure11.
in this example, the support structure of 11 implements a cylindrical band wrapped around a basic object, such as an actor 114. In some embodiments, the support structure of 11 is rigid and cannot bend to a large extent (e.g., relatively accurate/precise camera and/or motion capture program). In addition, the supportable structure 11 is semi-rigid and can be bent in a limited way.
in addition, the support structure 11 can have any number of colors, such as a series of colors that greatly contrast with each other. In banyan. 1, each supporting structure 11 has a series of alternating black andwhite square areas.
112 markers, in turn, are connected to a supporting structure 11, so that a pair of adjacent markers 112 can be separated from each other by a fixed or semi-fixed distance, and the supporting structure 11 along the axis marks the measurement throughthe. Each pair of 112 adjacent markers, in turn, can be separated by the same distance. For example, the logo 112 can form a supporting structure 11 to make the trademarks 112 evenly spaced. In some implementations, signs donot need to be evenly distributed, except, but there may still be other signs that do not change the distance greatly.
when a rigid supporting structure 11 is used, the distance between two adjacent trademarks 112 can be greatly fixed. Therefore, the distance remains constant when the underlying object moves 114 from a position toanother position.
When a semi-rigid supporting structure 11 is used, the distance between two adjacent trademarks 112 may not be fixed, but may be different within a limited range. Therefore, the distance may be within a certain range when the underlying object 114moves from one location to another. For example, the distance between two adjacent semi-rigid support structures 11 can be different, because the structure of 11 may be bent or twisted. In some embodiments, due to the strength of one or more application support structures, the quantity of possiblevariation is the observed distance between measuring and inputting 1 markers into the system to compensate for possible changes. Therefore, semi-rigid support members may be limited to a limited range, depending on the direction and magnitude of material support member countries' strength, such as variables, positions of themarks 112 movement.
in addition, the marker 112 may have one or more geometric shapes. For example, the symbol 112 can be realized with a circle, a triangle, a square, or a rectangle. In the example of Figure 1, each trademark 112 has the same geometry, and andis is implemented with dots.
besides having one or more shapes, the logo 112 can have any one or more colors, such as color, and the color of the support structure 11 with a large contrast (for example, the contrast of 4:1). In the example in fig. 1, a series of alternate contrasting colors (white and black) and a series of supporting structures (black and white) are marked with 112have.
in addition, in some embodiments, a line marks 112 the contrast that can be implemented (e.g., black and white). One of the advantages of using the contrast line sign 112 is that the production line can be used to form a bar codewhich, in turn, can be used to uniquely identify the sign 112. For example, it may include the correlation of the computer 124 index, especially the observation bar code of the trademark. When a bar code is taken by a camera, the index may access the identifywhich sign, which is designated by the code (for example, the identification sign can be placed on the elbow support structure inside the left elbow).
in addition, the sign 112 can be covered with or composed of a luminescent material, or it can emit light by itself, such as 112 marks, using light emitting diodes (LEDs). For example, infrared light, from the luminous sign 112, it can be seen that ITIS is largely completely black, and as a result, it provides invariance under lighting conditions.
Therefore, luminous or luminous signs may help to capture the action of a live performance at night, or more commonly, when an actor walks away, a thick shadow turns into light; Or the number of characters between the representatives of the actors in the dark area, and then into the light area.
indicates that 112 can be non-planar (e.g., protruding) with support structure 11, with support structure 11, or both. In banyan. The example in each trademark 112 1 realizes a point thatlies of the support structure 11 on the same plane. In addition, support structures 11 independent of individual trademarks 116 can be used.
in addition to supporting structures 11 and 112 marks, the 1 system also includes one or more? A camera 12 captures an image of 3d scanner, marking 112 performing a series of actions as an actor 114. In addition, the 1 system includes astorage medium-sized 122 connected to the camera 12 for digital recording of 3d scanner images taken by the camera 12.
the p>1 system also includes 124 a computer connected to the storage medium 122. As described in more detail below, the 124 computer can execute a motion capture program to extractthe motion of the underlying object, such as an actor, which marks the motion from frame to frame.
in operation, the actor 114 is placed around one or more cameras 12. 114 actors, in turn, wear a black tights, while the supporting structure 11 is placed around 114 actors' bodies to restore their bones. In addition, 116 personal markers can be placed directly on the actor's chest and back. (The support structure 11 may be alternately-except-placed around the actor's chest. )
Any logo 112 located on the supporting structure of 11, including zero, can be seen on each camera. For example, a support structure 11 may have 3 points 112, which are fully visible from each camera view. In another example, only the asingle flag 112 can be seen in the camera view. In some embodiments, no camera 12 needs 112 the same trademark to reconstruct the actor's virtual skeleton structure.
once the system 1 is established, the actor 114 can perform a series of actions. The supporting structure of 11 allows the marks 112 to 114 to have a substantially fixed relationship with actors and each other. In some embodiments, when the actor 114 moves, themarks 112 follow the movement of the actor 114 points, although it marks 112 hard attachment.
support the change of structure 11 112 marks, and then capture 12 the camera, digital recording storage medium 122. The recorded data can then be processed by the computer 124 using the motion capture program.
as mentioned above, figure. 2 shows a flowchart illustrating an example of a method 2 for capturing data frames by directional processing actions of a virtual skeleton structure of a derived object. Combined with the tracking object's bone structure and the motion between frames, the processing frame generatesinformation.
fig. 3 shows a flowchart illustrating an example of each virtual structure position of each frame supporting structure calculated by a method of 3. As shown in the figure, method 3 3 31 camera view issele.