please help me with this asap do it carefully and clearly plz.
# Import library import numpy as #(2 points) # Function register_Arun_3D # Input: Two 3x matrices of points to register: # fixed dataset F of N points and moving dataset M of N points # Output: R = 3x3 rotation matrix, t = 3x1 column vector # such that M = R@F+t # Note: the operator @ is used for matrix multiplication. def register_Arun_3D(F, M): Ensure the matrices F and have the same shape: same dimension and same amount of points to register if F._----- != M._____: #(5 points) raise Exception("The datasets do not have the same size") # Ensure the points have 3D coordinates num_rows, num_cols = #(5 points) if num_rows != 3: raise Exception("The datasets do not have a 3xN size") # (i) Find R # Step 1, Arun's algorithm # Find centroids of F and M column wise centroid_F = np.(F, axis=1) #(3 points) centroid_M = (M, axis-1) #(3 points) # Ensure F and M have same centroid ..reshape(-1, 1) #(5 points) ..reshape(-1, 1) #(5 points) FC = Mc = # Step 2, Arun's algorithm #Calculate the 3x3 matrix H H = @ np. #(5 points) # Step 3, Arun's algorithm # Find the SVD of H U, S, Vt = (H) #(7 points) # Step 4, Arun's algorithm # Calculate X X = ----TO T #(5 points) # Step 5, Arun's algorithm R = X # Calculate, det (x), the determinant of X if - ---(x) , >, = 0. Then +,e + 0 - MyP (17) Therefore, minimizing Sis equivalent to maximizing where, and are columns of U and respectively. Note that changing the sign of wory will not change H. Therefore, if X- F- VU' minimizes C. so does X'= V'U' where V-..-1 (18) Trace R9.4") - Trace (RH) (14) IX is a reflection, then X' is a rotation, and vice versa. Thus, if the SVD algorithm gives a solution X with det (X) = -1, we form where X'=V'U' which is the desired rotation. We mention, in passing, that the points are colinear, if and (11) only if, two of the three singular values of Hare equal. V. DEGENERACY: Nousy CASE Lemma: For any positive definite matrix A. and any ontho normal matrix B. If either {.) or ( } are coplanar, then it can readily be shown that the discussion on the coplanar case in Section IV is still valid, Trace (AA) 2 Trace (BAX'). except of course now the minimum of is no longer zero. Hence, Proof of Lemna: Leta, be theith column of A. Then if the SVD algorithm gives a reflection X-VU'we can form the desired rotation X = 'U'. A special case of interest is when N Trace (BAR) - Trace (ABA) 3. Then both ) and (4) are coplanar point sets. The situation we cannot handle is when the SVD algorithm gives - a(Ba). a solution X with det (X) = -1. and none of the singular values But, by the Schwarz inequality. of His zero. This means that neither ) nor fl) are coplanar. yet there is no rotation which yields a smaller then the reflection (Ba) s vaa)(B'Ba ) - 04. 1. This can happen only when the noise N, are very large. In that case, the least-squares solution is probably useless anyway. A bet- Hence, Trace (BA) S = Trace (AA). Q.ED ter approach would be to use a RANSAC-like technique (using 3 Let the SVD of H be: points at a time) to combat against outliers 151. H- UAV (12) VI SUMMARY OF ALGORITHIM where U and Vare 3 x 3 orthonormal matrices, and A is a 3 x 3 Using the procedure of Section II-A, we obtain diagonal matrix with nonnegative elements. Now let X-VU. X-YU(which is onthonormal). (13) 1) If det (X) = +1, then X is a rotation which is the desired We have solution XHVU'UAV 2) If det (X) = -1. then X is a reflection a) one of the singular values (s. say) of His zero. Then, the - VAR (15) desired rotation is found by forming which is symmetrical and positive definite. Therefore, from X-VU Lemma, for any 3 X 3 orthonormal matrix B. where V' is obtained from V by changing the sign of the 3rd col- Trace (H) : Truce (EXH) (16) b) None of the singular values of His zero. Then, conven- Thus, among all 3 x 3 or honormal matrices, X maximizes F of tional least squares solution is probably not appropriate. We go to (14). And if det (X) - +1.X is a rotation, which is what we want a RANSAC-like technique However, if det (X) = -1.X is a reflection, which is not what we want. Fortunately, this degenerate case usually does not occur. VII. COMPUTER TIME REQUIREMENTS We shall discuss the situation in some detail in the next two- Computer simulations have been carried out on a VAX 11/780 tions to compare the three algorithms (SVD, quaterion, iterative) with IV. DEGENERACY: NOISELESS CASE respect to time requirements. In each simulation, a set of 3-D points In) were generated. They are randomly distributed in a cube of Assume N0 in (I) for all L. Then, obviously there is a solu- size 6 x 6 x 6 with center at(0,0,0). Then it) were calculated tion R(which is a rotation, e., det (R) - +1) for which (1) by rotating (n) by as angle of 75 around an axis through the and (R) are congruent and hence - 0. From geometrical origin with direction cosines (0.6.0.7.0.39) followed by a trans- considerations, it is easy to see that there are tree possibilities, lation of (80, 60, 70), and finally by adding to each coordinate of 1) L.) are nor coplanar--Then, the rotation solution is unique. the molting points Gamian random noise with mean zero and Furthermore, there is no reflection X which can make ? -0. standard deviation 0 5. Then the algorithms were used to estimate Therefore, the SVD algorithm will give the desired solution Rand T. The CPU times used are listed in Table L. For the iterative 2) 7.) are coplanar but not colinear-There is a unique to algorithm, the number of iterations are given in parentheses. The tation as well as a unique reflection which will make -0. programs were written in C. The IMSL sobroutine package was TABLE 1 VAX 11/780 CPU TIME PER RUN IN M cusses an extension of the method to cover both translational and ro- tational movements. data, Method Used Index Terms --Digital image processing, fast Fourier transform, im- Number of Point age registration. Image sequence analysis, motion estimation. Correspondences SVD Quatemion Iterative 3 54.6 26.6 126.8 (25) 1. INTRODUCTION 2 41.6 32.4 108.2 (12) Let us consider a plane image performing rigid movements of 11 37.0 41.0 103.2 (8) translation and rotation within a rectangular domain representing 16 39.4 45.6 94.2 (5) 20 the observed field. Let the image he defined by a density function 40,4 452 135.0 (6) 44.2 111.0 (6) vanishing outside a region A, whose position varies with time, but is always fully contained inside C. The limitations which arise when parts of the image leave the observed field will be mentioned later on. used in finding the SVD (subroutine LSVDF) and in doing the ei Lct (.) represent the image at a reference time, 1 = 0, and gen analysis (subroutine EIGRS) for the quaternion method. For ,(x,y) be the present image, which is but a replica of so(x, y) the iterative method, the initial guess solution was zero in all cases translated by (to) and rotated by : We observe that the computer time requirements of the SVD and the quaternion algorithms are comparable, while the time for the $(xy) - 4o, v). (1). iterative method is much longer. However, in the iterative method, where the solutions were calculated to 7-digit accuracy. If we can accept - cos sin B 10 percent accuracy, then the number of iterations are reduced by (2) a factor of 2 to 3. Furthermore, the rate of convergence can be - 3 sin fo + cos de increased by overrelaxation. In order to realign images 5o and s, it is first necessary to deter- mine the translation vector (.yo) and the rotation from the REFERENCES information provided by so(t. y) and (x. y). Several image reg. istration algorithms are known for excellent review papers see 11)- II S. D. Blostein and T. S. Huang, ""Estimating 3-D motion from range 16 in Proc. Ist Conf. Aricial Intelligence Applications. Denver. The present correspondence is meant as a contribution in the CO. Dec. 1984. pp. 246-250. 121 D. Cyganski and A. Or Applications of tenor theory to obietveloped with reference to purely translatory displacements. Ac context of the phase correlation technique 171. [8, which was de recognition and orientation determination," IEEE Trans Pantern Anal. Machine Intell., vol. PAMI-7. PP. 663-673. Nov. 1985, cording to 17). [8). let S.(.) and S, (.) be the Fourier trans 13] T. S. Huang, S. D. Blostein, and E. A. Margerum, "Least-squares forms of so(x,y) and (x.y). Since in the case of pure translations estimation of motion parameters from 3-D point correspondences.in $,(x, y) = (x - Xoy - Yo). it follows that Proe IEEE Conf. Computer Vision and Pantern Recognition, Miami Beach, FL, June 24-26, 1986. S. (8.) - S.C. ). (3) 141 O. D. Fauperus and M. Heben. "A 3-D recognition and positioning Therefore, by inverse transforming the ratio of the cross-power algorithm using geometrical matching between primitive surfaces, in Proc. In Joint Conf Arificial Intelligence, Karlshrue, West Ger spectrum of s, and to its magnitude. S/IS.SI - exp masy. Aeg. 1983, pp. 996-1002 (-12 + )), a Dirac b-distribution centered on (vo) 15 M. Fischler and Bolles. "Random sample consenses: A paradigm for is obtained. In practice continuous transforms are replaced by finite model fitting with applications to image analysis and automated car ones, and by inverse transformation a unity pulse centered on (o. tography." Com ACM, vol. 24, 106, June 1981. Ya) is obtained, so that the translation is immediately determined In this correspondence the principles of a generalization of the phase correlation method for the registration of rotated and trans- lated images (9) are briefly recalled and the corresponding numer Registration of Translated and Rotated Images Using ical algorithm is presented. The effectiveness of the procedure is then illustrated by means of simple experiments. Finally, the edge Finite Fourier Transforms effects which arise when the image completely fills the observed field Care pointed out E DE CASTRO AND C. MORANDI This correspondence is a part of research aiming at the imple- mentation of an image stabilization system [10for the observation AbstractA well-known method for image registration is based on of the human retina, in which the image viewed by the TV camera may be observed on a monitor free of the unavoidable spontaneous a conventional correlation between phase-only, or whitened, versions of the two images to be realigned. The method, covering rigid trans- movements, which do not allow any automatic analysis of dynamic Iational movements, is characterized by an outstanding robustness effects, such as the pulsations frequently observed in blood vessels. agalast correlated male and disturbances, such as these encountered The program is came out in cooperation with the Ophthalmolog- with menuniform, time varying illumination. This correspondence die ical Clinic of the University of Bologna and the IBM Research Center in Rome. Manuscript received October 11, 1984. Recommended for acceptance II. THEORY by S. W. Zucker E. De Castro was with the Dipartimento di Elettronka, Informatica e If(x,y) is a translated and rotated replica of do(. y). face Sistemistica. Universit di Bologna. Viale Risorgimento 2. 40136 Bo (1) according to the Fourier Shift Theorem and the Fourier Ro logna, Italy tation Theorem [11] their transforms are related by C. Morandi is with the Dipanimento di Elettronice ed Automatica, Universit di Ancona, via Breece Bianche. 60100 Ancona, Italy $(&) - S. coss + sin 8. IEEE Log Number 8715507 - sino + cos ) (4) # Import library import numpy as #(2 points) # Function register_Arun_3D # Input: Two 3x matrices of points to register: # fixed dataset F of N points and moving dataset M of N points # Output: R = 3x3 rotation matrix, t = 3x1 column vector # such that M = R@F+t # Note: the operator @ is used for matrix multiplication. def register_Arun_3D(F, M): Ensure the matrices F and have the same shape: same dimension and same amount of points to register if F._----- != M._____: #(5 points) raise Exception("The datasets do not have the same size") # Ensure the points have 3D coordinates num_rows, num_cols = #(5 points) if num_rows != 3: raise Exception("The datasets do not have a 3xN size") # (i) Find R # Step 1, Arun's algorithm # Find centroids of F and M column wise centroid_F = np.(F, axis=1) #(3 points) centroid_M = (M, axis-1) #(3 points) # Ensure F and M have same centroid ..reshape(-1, 1) #(5 points) ..reshape(-1, 1) #(5 points) FC = Mc = # Step 2, Arun's algorithm #Calculate the 3x3 matrix H H = @ np. #(5 points) # Step 3, Arun's algorithm # Find the SVD of H U, S, Vt = (H) #(7 points) # Step 4, Arun's algorithm # Calculate X X = ----TO T #(5 points) # Step 5, Arun's algorithm R = X # Calculate, det (x), the determinant of X if - ---(x) , >, = 0. Then +,e + 0 - MyP (17) Therefore, minimizing Sis equivalent to maximizing where, and are columns of U and respectively. Note that changing the sign of wory will not change H. Therefore, if X- F- VU' minimizes C. so does X'= V'U' where V-..-1 (18) Trace R9.4") - Trace (RH) (14) IX is a reflection, then X' is a rotation, and vice versa. Thus, if the SVD algorithm gives a solution X with det (X) = -1, we form where X'=V'U' which is the desired rotation. We mention, in passing, that the points are colinear, if and (11) only if, two of the three singular values of Hare equal. V. DEGENERACY: Nousy CASE Lemma: For any positive definite matrix A. and any ontho normal matrix B. If either {.) or ( } are coplanar, then it can readily be shown that the discussion on the coplanar case in Section IV is still valid, Trace (AA) 2 Trace (BAX'). except of course now the minimum of is no longer zero. Hence, Proof of Lemna: Leta, be theith column of A. Then if the SVD algorithm gives a reflection X-VU'we can form the desired rotation X = 'U'. A special case of interest is when N Trace (BAR) - Trace (ABA) 3. Then both ) and (4) are coplanar point sets. The situation we cannot handle is when the SVD algorithm gives - a(Ba). a solution X with det (X) = -1. and none of the singular values But, by the Schwarz inequality. of His zero. This means that neither ) nor fl) are coplanar. yet there is no rotation which yields a smaller then the reflection (Ba) s vaa)(B'Ba ) - 04. 1. This can happen only when the noise N, are very large. In that case, the least-squares solution is probably useless anyway. A bet- Hence, Trace (BA) S = Trace (AA). Q.ED ter approach would be to use a RANSAC-like technique (using 3 Let the SVD of H be: points at a time) to combat against outliers 151. H- UAV (12) VI SUMMARY OF ALGORITHIM where U and Vare 3 x 3 orthonormal matrices, and A is a 3 x 3 Using the procedure of Section II-A, we obtain diagonal matrix with nonnegative elements. Now let X-VU. X-YU(which is onthonormal). (13) 1) If det (X) = +1, then X is a rotation which is the desired We have solution XHVU'UAV 2) If det (X) = -1. then X is a reflection a) one of the singular values (s. say) of His zero. Then, the - VAR (15) desired rotation is found by forming which is symmetrical and positive definite. Therefore, from X-VU Lemma, for any 3 X 3 orthonormal matrix B. where V' is obtained from V by changing the sign of the 3rd col- Trace (H) : Truce (EXH) (16) b) None of the singular values of His zero. Then, conven- Thus, among all 3 x 3 or honormal matrices, X maximizes F of tional least squares solution is probably not appropriate. We go to (14). And if det (X) - +1.X is a rotation, which is what we want a RANSAC-like technique However, if det (X) = -1.X is a reflection, which is not what we want. Fortunately, this degenerate case usually does not occur. VII. COMPUTER TIME REQUIREMENTS We shall discuss the situation in some detail in the next two- Computer simulations have been carried out on a VAX 11/780 tions to compare the three algorithms (SVD, quaterion, iterative) with IV. DEGENERACY: NOISELESS CASE respect to time requirements. In each simulation, a set of 3-D points In) were generated. They are randomly distributed in a cube of Assume N0 in (I) for all L. Then, obviously there is a solu- size 6 x 6 x 6 with center at(0,0,0). Then it) were calculated tion R(which is a rotation, e., det (R) - +1) for which (1) by rotating (n) by as angle of 75 around an axis through the and (R) are congruent and hence - 0. From geometrical origin with direction cosines (0.6.0.7.0.39) followed by a trans- considerations, it is easy to see that there are tree possibilities, lation of (80, 60, 70), and finally by adding to each coordinate of 1) L.) are nor coplanar--Then, the rotation solution is unique. the molting points Gamian random noise with mean zero and Furthermore, there is no reflection X which can make ? -0. standard deviation 0 5. Then the algorithms were used to estimate Therefore, the SVD algorithm will give the desired solution Rand T. The CPU times used are listed in Table L. For the iterative 2) 7.) are coplanar but not colinear-There is a unique to algorithm, the number of iterations are given in parentheses. The tation as well as a unique reflection which will make -0. programs were written in C. The IMSL sobroutine package was TABLE 1 VAX 11/780 CPU TIME PER RUN IN M cusses an extension of the method to cover both translational and ro- tational movements. data, Method Used Index Terms --Digital image processing, fast Fourier transform, im- Number of Point age registration. Image sequence analysis, motion estimation. Correspondences SVD Quatemion Iterative 3 54.6 26.6 126.8 (25) 1. INTRODUCTION 2 41.6 32.4 108.2 (12) Let us consider a plane image performing rigid movements of 11 37.0 41.0 103.2 (8) translation and rotation within a rectangular domain representing 16 39.4 45.6 94.2 (5) 20 the observed field. Let the image he defined by a density function 40,4 452 135.0 (6) 44.2 111.0 (6) vanishing outside a region A, whose position varies with time, but is always fully contained inside C. The limitations which arise when parts of the image leave the observed field will be mentioned later on. used in finding the SVD (subroutine LSVDF) and in doing the ei Lct (.) represent the image at a reference time, 1 = 0, and gen analysis (subroutine EIGRS) for the quaternion method. For ,(x,y) be the present image, which is but a replica of so(x, y) the iterative method, the initial guess solution was zero in all cases translated by (to) and rotated by : We observe that the computer time requirements of the SVD and the quaternion algorithms are comparable, while the time for the $(xy) - 4o, v). (1). iterative method is much longer. However, in the iterative method, where the solutions were calculated to 7-digit accuracy. If we can accept - cos sin B 10 percent accuracy, then the number of iterations are reduced by (2) a factor of 2 to 3. Furthermore, the rate of convergence can be - 3 sin fo + cos de increased by overrelaxation. In order to realign images 5o and s, it is first necessary to deter- mine the translation vector (.yo) and the rotation from the REFERENCES information provided by so(t. y) and (x. y). Several image reg. istration algorithms are known for excellent review papers see 11)- II S. D. Blostein and T. S. Huang, ""Estimating 3-D motion from range 16 in Proc. Ist Conf. Aricial Intelligence Applications. Denver. The present correspondence is meant as a contribution in the CO. Dec. 1984. pp. 246-250. 121 D. Cyganski and A. Or Applications of tenor theory to obietveloped with reference to purely translatory displacements. Ac context of the phase correlation technique 171. [8, which was de recognition and orientation determination," IEEE Trans Pantern Anal. Machine Intell., vol. PAMI-7. PP. 663-673. Nov. 1985, cording to 17). [8). let S.(.) and S, (.) be the Fourier trans 13] T. S. Huang, S. D. Blostein, and E. A. Margerum, "Least-squares forms of so(x,y) and (x.y). Since in the case of pure translations estimation of motion parameters from 3-D point correspondences.in $,(x, y) = (x - Xoy - Yo). it follows that Proe IEEE Conf. Computer Vision and Pantern Recognition, Miami Beach, FL, June 24-26, 1986. S. (8.) - S.C. ). (3) 141 O. D. Fauperus and M. Heben. "A 3-D recognition and positioning Therefore, by inverse transforming the ratio of the cross-power algorithm using geometrical matching between primitive surfaces, in Proc. In Joint Conf Arificial Intelligence, Karlshrue, West Ger spectrum of s, and to its magnitude. S/IS.SI - exp masy. Aeg. 1983, pp. 996-1002 (-12 + )), a Dirac b-distribution centered on (vo) 15 M. Fischler and Bolles. "Random sample consenses: A paradigm for is obtained. In practice continuous transforms are replaced by finite model fitting with applications to image analysis and automated car ones, and by inverse transformation a unity pulse centered on (o. tography." Com ACM, vol. 24, 106, June 1981. Ya) is obtained, so that the translation is immediately determined In this correspondence the principles of a generalization of the phase correlation method for the registration of rotated and trans- lated images (9) are briefly recalled and the corresponding numer Registration of Translated and Rotated Images Using ical algorithm is presented. The effectiveness of the procedure is then illustrated by means of simple experiments. Finally, the edge Finite Fourier Transforms effects which arise when the image completely fills the observed field Care pointed out E DE CASTRO AND C. MORANDI This correspondence is a part of research aiming at the imple- mentation of an image stabilization system [10for the observation AbstractA well-known method for image registration is based on of the human retina, in which the image viewed by the TV camera may be observed on a monitor free of the unavoidable spontaneous a conventional correlation between phase-only, or whitened, versions of the two images to be realigned. The method, covering rigid trans- movements, which do not allow any automatic analysis of dynamic Iational movements, is characterized by an outstanding robustness effects, such as the pulsations frequently observed in blood vessels. agalast correlated male and disturbances, such as these encountered The program is came out in cooperation with the Ophthalmolog- with menuniform, time varying illumination. This correspondence die ical Clinic of the University of Bologna and the IBM Research Center in Rome. Manuscript received October 11, 1984. Recommended for acceptance II. THEORY by S. W. Zucker E. De Castro was with the Dipartimento di Elettronka, Informatica e If(x,y) is a translated and rotated replica of do(. y). face Sistemistica. Universit di Bologna. Viale Risorgimento 2. 40136 Bo (1) according to the Fourier Shift Theorem and the Fourier Ro logna, Italy tation Theorem [11] their transforms are related by C. Morandi is with the Dipanimento di Elettronice ed Automatica, Universit di Ancona, via Breece Bianche. 60100 Ancona, Italy $(&) - S. coss + sin 8. IEEE Log Number 8715507 - sino + cos ) (4)
