Notes on the FFT C. S. Burrus Department of Electrical and Computer Engineering Rice University, Houston, TX 77251-1892, e-mail: csb@rice.edu This is a note describing results on efficient algorithms to calculate the discrete Fourier transform (DFT). The purpose is to report work done at Rice University, but other contributions used by the DSP research group at Rice are also cited. Perhaps the most interesting is the discovery that the Cooley-Tukey FFT was described by Gauss in 1805 [1]. That gives some indication of the age of research on the topic, and the fact that a recently compiled bibliography [2] on efficient algorithms contains over 3400 entries indicates its volume. An expanded version of this bibliography is published as a book [2] with the references in a data base on a disk. Four IEEE Press reprint books contain papers on the FFT [3,4,5,6]. There are several books [7,8,9,10,11,12,13,14,15] that give a good modern theoretical background for this work, one book [16] that gives the basic theory plus both FORTRAN and TMS 320 assembly language programs, and another book [17] that contains a chapter on advanced FFT topics. The history of the FFT is outlined in [18,1] and excellent survey articles can be found in [19,20]. The foundation of much of the modern work on efficient algorithms was done by S. Winograd. This can be found in [21,22,23]. An outline and discussion of his theorems can be found in [17] as well as [7,8,9,10]. Efficient FFT algorithms for length-2M were described by Gauss and discovered in modern times by Cooley and Tukey [24]. These have been highly developed and good examples of FORTRAN programs can be found in [16]. Several new algorithms have been published that require the least known amount of total arithmetic [25,26,27,28 ,29]. Of these, the split-radix FFT [26,27,30,31] seems to have the best structure for programming, and an efficient program has been written [32] to implement it. A mixture of decimation-in-time and decimation-in-frequency with very good efficiency is given in [33 ]. Theoretical bounds on the number of multiplications required for the FFT based on Winograd's theories are given in [10,34]. Schemes for calculating an in-place, in-order radix-2 FFT are given in [35, 36,37]. Discussion of various forms of unscramblers is given in [38,39,40,41,42,43,44,45,46]. A discussion of the relation of the computer architecture, algorithm and compiler can be found in [47,48]. The "other" FFT is the prime factor algorithm (PFA) which uses an index map originally developed by Thomas and by Good. The theory of the PFA was derived in [49] and further developed and an efficient in-order and in-place program given in [50,16]. More results on the PFA are given in [51,52,37,53,54]. A method has been developed to use dynamic programming to design optimal FFT programs that minimize the number of additions and data transfers as well as multiplications [55]. This new approach designs custom algorithms for a particular computer architecture. An efficient and practical development of Winograd's ideas has given a design method that does not require the rather difficult Chinese remainder theorem [17,56] for short prime length FFT's. These ideas have been used to design modules of length 11,13, 17, 19, and 25 [57]. Other methods for designing short DFT's can be found in [58,59]. A use of these ideas with distributed arithmetic and table look-up rather than multiplication is given in [60]. A program that implements the nested Winograd Fourier transform algorithm (WFTA) is given in [7] but it has not proven as fast or as versatile as the PFA [50]. An interesting use of the PFA was announced [61] in searching for large prime numbers. These efficient algorithms can not only be used on DFT's but on other transforms with a similar structure. They have been applied to the discrete Hartley transform [62,63] and the discrete cosine transform [29,64,65]. The fast Hartley transform has been proposed as a superior method for real data analysis but that has been shown not to be the case. A well-designed real-data FFT [66] is always as good as or better than a well-designed Hartley transform [62,67,68,69]. The Bruun algorithm [70,71] also looks promising for real data applications as does the Rader-Brenner algorithm [72,73,69]. A novel approach to calculating the inverse DFT is given in [74]. General length algorithms include [75,76,77]. For lengths that are not highly composite or prime, the chirp z-transform in a good candidate [16,78] for longer lengths and an efficient order-N^2 algorithm called the QFT [79,80,81] for shorter lengths. A method which automatically generates near-optimal prime length Winograd based programs has been given in [56,82,83,84,85]. Special methods are available for very long lengths [86,87]. The use of the FFT to calculate discrete convolution was one of its earliest uses. Although the more direct rectangular transform [88] would seem to be more efficient, use of the FFT or PFA is still probably the fastest method on a general purpose computer or DSP chip [89,66,67, 90] although the use of distributed arithmetic [60] or number theoretic transforms [91] with special hardware may be even faster. Special algorithms for use with the short-time Fourier transform [92] and for the calculation of a few DFT values [93,94,95] have also been developed. An excellent analysis of efficient programming the FFT on DSP microprocessors is given in [96,48]. Formulations of the DFT in terms of tensor or Kronecker products look promising for developing algorithms for parallel and vector computer architectures [97,11,98,99,100, 101,102]. The study of efficient algorithms not only has a long history and large bibliography, it is still an exciting research field where new results are used in practical applications. References [1] M. T. Heideman, D. H. Johnson, and C. S. Burrus, "Gauss and the history of the FFT," IEEE Acoustics, Speech, and Signal Processing Magazine, vol. 1, pp. 14-21, October 1984. also in IEEE Press FFT Reprints, by P. Duhamel, 1995. [2] H. V. Sorensen, C. S. Burrus, and M. T. Heideman, Fast Fourier Transform Database. Boston: PWS Publishing, 1995. Update of Technical Report TR-8402. [3] L. R. Rabiner and C. M. Rader, eds., Digital Signal Processing, selected reprints. New York: IEEE Press, 1972. [4] DSP Committee, ed., Digital Signal Processing II, selected reprints. New York: IEEE Press, 1979. [5] DSP Committee, ed., Programs for Digital Signal Processing. New York: IEEE Press, 1979. [6] P. Duhamel, ed., Papers on the Fast Fourier Transform. New York: IEEE Press, to appear in 1995. [7] J. H. McClellan and C. M. Rader, Number Theory in Digital Signal Processing. Englewood Cliffs, NJ: Prentice-Hall, Inc., 1979. [8] H. J. Nussbaumer, Fast Fourier Transform and Convolution Algorithms. Heidelberg, Germany: Springer-Verlag, second ed., 1981, 1982. [9] R. E. Blahut, Fast Algorithms for Digital Signal Processing. Reading, MA: Addison-Wesley, Inc., 1984. [10] M. T. Heideman, Multiplicative Complexity, Convolution, and the DFT. Springer-Verlag, 1988. [11] R. Tolimieri, M. An, and C. Lu, Algorithms for Discrete Fourier Transform and Convolution. New York: Springer-Verlag, 1989. [12] D. G. Myers, Digital Signal Processing, Efficient Convolution and Fourier Transform Techniques. Sydney, Australia: Prentice-Hall, 1990. [13] R. E. Blahut, Algebraic Methods for Signal Processing and Communications Coding. New York: Springer-Verlag, 1992. [14] W. L. Briggs and V. E. Henson, The DFT: An Owner's Manual for the Discrete Fourier Trnasform. Philadelphia: SIAM, 1995. [15] W. W. Smith and J. M. Smith, Handbook of Real-Time Fast Fourier Transforms. New York: IEEE Press, 1995. [16] C. S. Burrus and T. W. Parks, DFT/FFT and Convolution Algorithms. New York: John Wiley & Sons, 1985. [17] J. S. Lim and A. V. Oppenheim, Advanced Topics in Signal Processing, ch. 4. Prentice-Hall, 1988. [18] J. W. Cooley, "How the FFT gained acceptance," IEEE Signal Processing Magazine, vol. 9, pp. 10-13, January 1992. Also presented at the ACM Conference on the History of Scientific and Numerical Computation, Princeton, NJ, May 1987 and published in: A History of Scientific Computing, edited by S. G. Nash, Addison-Wesley, 1990, pp. 133-140. [19] P. Duhamel and M. Vetterli, "Fast Fourier transforms: a tutorial review and a state of the art," Signal Processing, vol. 19, pp. 259-299, April 1990. [20] J. W. Cooley, "The structure of FFT algorithms," April 1990. Notes for a Tutorial at IEEE ICASSP-90. [21] S. Winograd, "On computing the discrete Fourier transform," Mathematics of Computation, vol. 32, pp. 175-199, January 1978. [22] S. Winograd, "On the multiplicative complexity of the discrete Fourier transform," Advances in Mathematics, vol. 32, pp. 83-117, May 1979. also in [7]. [23] S. Winograd, Arithmetic Complexity of Computation. SIAM CBMS-NSF Series, No. 33, Philadelphia: SIAM, 1980. [24] J. W. Cooley and J. W. Tukey, "An algorithm for the machine calculation of complex Fourier series," Math. Computat., vol. 19, pp. 297-301, 1965. [25] R. Yavne, "An economical method for calculating the discrete Fourier transform," in Proceedings of the Fall Joint Computer Conference, pp. 115-125, 1968. [26] P. Duhamel and H. Hollmann, "Split radix FFT algorithm," Electronic Letters, vol. 20, pp. 14-16, January 5 1984. [27] P. Duhamel, "Implementation of `split-radix' FFT algorithms for complex, real, and real-symmetric data," IEEE Trans. on ASSP, vol. 34, pp. 285-295, April 1986. A shorter version appeared in the ICASSP-85 Proceedings, p. 20.6, March 1985. [28] J. B. Martens, "Recursive cyclotomic factorization - a new algorithm for calculating the discrete Fourier transform," IEEE Trans. on ASSP, vol. 32, pp. 750-762, August 1984. [29] M. Vetterli and H. J. Nussbaumer, "Simple FFT and DCT algorithms with reduced number of operations," Signal Processing, vol. 6, pp. 267-278, August 1984. [30] M. Vetterli and P. Duhamel, "Split-radix algorithms for length - pm DFT's," IEEE Trans. on ASSP, vol. 37, pp. 57-64, January 1989. Also, ICASSP-88 Proceedings, pp. 1415-1418, April 1988. [31] R. Stasinski, "The techniques of the generalized fast Fourier transform algorithm," IEEE Transactions on Signal Processing, vol. 39, pp. 1058-1069, May 1991. [32] H. V. Sorensen, M. T. Heideman, and C. S. Burrus, "On calculating the split-radix FFT," IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 34, pp. 152-156, February 1986. [33] A. Saidi, "Decimation-in-time-frequency FFT algorithm," in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, (IEEE ICASSP-94, Adelaide, Australia), pp. III:453-456, April 19-22 1994. [34] M. T. Heideman and C. S. Burrus, "On the number of multiplications necessary to compute a length-2n DFT," IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 34, pp. 91-95, February 1986. [35] J. K. Beard, "An in-place, self-reordering FFT," in Proceedings of the ICASSP-78, (Tulsa), pp. 632- 633, April 1978. [36] H. W. Johnson and C. S. Burrus, "An in-place, in-order radix-2 FFT," in ICASSP-84 Proceedings, p. 28A.2, March 1984. [37] C. Temperton, "Self-sorting in-place fast Fourier transforms," SIAM Journal of Sci. Statist. Comput., vol. 12, no. 4, pp. 808-823, 1991. [38] C. S. Burrus, "Unscrambling for fast DFT algorithms," IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 36, pp. 1086-1087, July 1988. [39] P. R"osel, "Timing of some bit reversal algorithms," Signal Processing, vol. 18, pp. 425-433, December 1989. [40] J. Jeong and W. J. Williams, "A fast recursive bit-reversal algorithm," in Proceedings of the ICASSP- 90, (Albuquerque, NM), pp. 1511-1514, April 1990. [41] D. M. W. Evans, "A second improved digit-reversal permutation algorithm for fast transforms," IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 37, pp. 1288-1291, August 1989. [42] J. J. Rodriguez, "An improved FFT digit-reversal algorithm," IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 37, pp. 1298-1300, August 1989. [43] J. S. Walker, "A new bit reversal algorithm," IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 38, pp. 1472-1473, August 1990. [44] A. A. Yong, "A better FFT bit-reversal algorithm without tables," IEEE Transactions on Signal Processing, vol. 39, pp. 2365-2367, October 1991. [45] D. Sundararajan, M. O. Ahamad, and M. N. S. Swamy, "A fast FFT bit-reversal algorithm," IEEE Transactions on Circuits and Systems, II, vol. 41, pp. 701-703, October 1994. [46] J. M. Rius and R. De Porrata-Doria, "New FFT bit-reversal algorithm," IEEE Transactions on Signal Processing, vol. 43, pp. 991-994, April 1995. [47] L. R. Morris, Digital Signal Processing Software. Toronto, Canada: DSPSW, Inc., 1982, 1983. [48] R. Meyer and K. Schwarz, "FFT implementation on DSP-chips," Sept. 18 1990. preprint. [49] D. P. Kolba and T. W. Parks, "A prime factor FFT algorithm using high speed convolution," IEEE Trans. on ASSP, vol. 25, pp. 281-294, August 1977. also in [7]. [50] C. S. Burrus and P. W. Eschenbacher, "An in-place, in-order prime factor FFT algorithm," IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 29, pp. 806-817, August 1981. Reprinted in it DSP Software, by L.R. Morris, 1983 and IEEE Press FFT Reprints, by P. Duhamel, 1995. [51] C. Temperton, "A self-sorting in-place prime factor real/half-complex FFT algorithm," Journal of Computational Physics, vol. 75, pp. 199-216, 1988. [52] C. Temperton, "Nesting strategies for prime factor FFT algorithms," Journal of Computational Physics, vol. 82, pp. 247-268, June 1989. [53] C. Temperton, "A generalized prime factor FFT algorithm for any n = 2p 3q5r," SIAM Journal of Sci. Stat. Comp., 1992. to appear. [54] R. Stasinski, "Prime factor DFT algorithms for new small-N DFT modules," IEE Proceedings, Part G, vol. 134, no. 3, pp. 117-126, 1987. [55] H. W. Johnson and C. S. Burrus, "The design of optimal DFT algorithms using dynamic programming," IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 31, pp. 378-387, April 1983. [56] H. W. Johnson and C. S. Burrus, "On the structure of efficient DFT algorithms," IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 33, pp. 248-254, February 1985. [57] H. W. Johnson and C. S. Burrus, "Large DFT modules: N = 11, 13, 17, 19, and 25," Tech. Rep. 8105, Department of Electrical Engineering, Rice University, Houston, TX 77251-1892, 1981. [58] C. Temperton, "A new set of minimum-add small-n rotated DFT modules," Journal of Computational Physics, vol. 75, pp. 190-198, 1988. [59] C. Lu, J. W. Cooley, and R. Tolimieri, "FFT algorithms for prime transform sizes and their implementations on VAX, IBM3090VF, and IBM RS/6000," IEEE Transactions on Signal Processing, vol. 41, pp. 638-648, February 1993. [60] S. Chu and C. S. Burrus, "A prime factor FFT algorithm using distributed arithmetic," IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 30, pp. 217-227, April 1982. [61] K. T. Chen, "A new record: the largest known prime number," IEEE Spectrum, vol. 27, p. 47, February 1990. [62] H. V. Sorensen, D. L. Jones, C. S. Burrus, and M. T. Heideman, "On computing the discrete Hartley transform," IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 33, pp. 1231-1238, October 1985. [63] R. N. Bracewell, The Hartley Transform. Oxford Press, 1986. [64] F. M. Wang and P. Yip, "Fast prime factor decomposition algorithms for a family of discrete trigonometric transforms," Circuits, Systems, and Signal Processing, vol. 8, no. 4, pp. 401-419, 1989. [65] K. R. Rao and P. Yip, Discrete Cosine Transform: Algorithms, Advantages, Applications. San Diego, CA: Academic Press, 1990. [66] H. V. Sorensen, D. L. Jones, M. T. Heideman, and C. S. Burrus, "Real valued fast Fourier transform algorithms," IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 35, pp. 849-863, June 1987. also in IEEE Press FFT Reprints, by P. Duhamel, 1995. [67] P. Duhamel and M. Vetterli, "Improved Fourier and Hartley transfrom algorithms, application to cyclic convolution of real data," IEEE Trans. on ASSP, vol. 35, pp. 818-824, June 1987. [68] M. Popovic and D. Sevic, "A new look at the comparison of the fast Hartley and Fourier transforms," IEEE Transactions on Signal Processing, vol. 42, pp. 2178-2182, August 1994. [69] P. R. Uniyal, "Transforming real-valued sequences: fast Fourier versus fast Hartley transform algorithms," IEEE Transactions on Signal Processing, vol. 42, pp. 3249-3254, November 1994. [70] G. Bruun, "Z-transform DFT filters and FFTs," IEEE Transactions on ASSP, vol. 26, pp. 56-63, February 1978. [71] R. Storn, "On the Bruun algorithm and its inverse," Frequenz, vol. 46, pp. 110-116, 1992. a German journal. [72] C. M. Rader and N. M. Brenner, "A new principle for fast Fourier transformation," IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. ASSP-24, pp. 264-266, June 1976. [73] K. M. Cho and G. C. Temes, "Real-factor FFT algorithms," in Proceedings of IEEE ICASSP-78, (Tulsa, OK), pp. 634-637, April 1978. [74] P. Duhamel, B. Piron, and J. M. Etcheto, "On computing the inverse DFT," IEEE Transactions on ASSP, vol. 36, pp. 285-286, February 1978. [75] R. Singleton, "An algorithm for computing the mixed radix fast Fourier transform," IEEE Transactions on Audio and Electroacoustics, vol. AU-17, pp. 93-103, June 1969. [76] J. A. Glassman, "A generalization of the fast Fourier transform," IEEE Transactions on Computers, vol. C-19, pp. 105-116, Feburary 1970. [77] W. E. Ferguson, Jr., "A simple derivation of Glassman general-n fast Fourier transform," Comput. and Math. with Appls., vol. 8, no. 6, pp. 401-411, 1982. Also, in Report AD-A083811, NTIS, Dec. 1979. [78] L. Rabiner, R. Schafer, and C. Rader, "The chirp z-transform algorithm," IEEE Transactions on Audio Electroacoustics, vol. AU-17, pp. 86-92, June 1969. [79] G. A. Sitton, "The QFT algorithm," unpublished paper, 1985. [80] H. Guo, G. A. Sitton, and C. S. Burrus, "The quick discrete Fourier transform," in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, (IEEE ICASSP-94, Adelaide, Australia), pp. III:445-448, April 19-22 1994. [81] H. Guo, G. A. Sitton, and C. S. Burrus, "The quick Fourier transform, an FFT based on symmetries," IEEE Transactions on Signal Processing, submitted Oct. 1994. [82] I. W. Selesnick and C. S. Burrus, "Automating the design of prime length FFT programs," in Proceedings of the IEEE International Symposium on Circuits and Systems, (ISCAS-92, San Diego, CA), pp. 133-136, May 1992. [83] I. W. Selesnick and C. S. Burrus, "Multidimensional mapping techniques for convolution," in Proceedings of the IEEE International Conference on Signal Processing, (IEEE ICASSP-93, Minneapolis), pp. III-288-291, April 1993. [84] I. W. Selesnick and C. S. Burrus, "Automatic generation of prime length FFT programs," IEEE Transactions on Signal Processing, to appear. 1995. [85] I. W. Selesnick and C. S. Burrus, "Extending Winograd's small convolution algorithm to longer lengths," in Proceedings of the IEEE International Symposium on Circuits and Systems, (IEEE ISCAS- 94, London), pp. 2.449-452, June 30 1994. [86] W. K. Hocking, "Performing Fourier transforms on extremely long data streams," Computers in Physics, vol. 3, pp. 59-65, January 1989. [87] R. Stasinski, "Extending sizes of effective convolution algorithms," Electronics Letters, vol. 26, no. 19, pp. 1602-1604, 1990. [88] R. C. Agarwal and J. W. Cooley, "New algorithms for digital convolution," IEEE Trans. on ASSP, vol. 25, pp. 392-410, October 1977. [89] I. Pitas and C. S. Burrus, "Time and error analysis of digital convolution by rectangular transforms," Signal Processing, vol. 5, pp. 153-162, March 1983. [90] R. Meyer, R. Reng, and K. Schwarz, "Convolution algorithms on DSP processors," in Proceedings of the ICASSP-91, (Toronto, Canada), pp. 2193-2196, May 1991. [91] R. C. Agarwal and C. S. Burrus, "Number theoretic transforms to implement fast digital convolution," Proceedings of the IEEE, vol. 63, pp. 550-560, April 1975. also in IEEE Press DSP Reprints II, 1979. [92] H. V. Sorensen and C. S. Burrus, "Efficient computation of the short-time FFT," in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, (New York), pp. 1894- 1897, April 1988. [93] H. V. Sorensen, C. S. Burrus, and D. L. Jones, "A new efficient algorithm for computing a few DFT points," in Proceedings of the IEEE International Symposium on Circuits and Systems, (Espoo, Finland), pp. 1915-1918, June 1988. [94] C. Roche, "A split-radix partial input/output fast Fourier transform algorithm," IEEE Transactions on Signal Processing, vol. 40, pp. 1273-1276, May 1992. [95] H. V. Sorensen and C. S. Burrus, "Efficient computation of the DFT with only a subset of input or output points," IEEE Transactions on Signal Processing, vol. 41, pp. 1184-1200, March 1993. [96] R. Meyer and K. Schwarz, "FFT implementation on DSP-chips _ theory and practice," in Proceedings of the ICASSP-90, (Albuquerque, NM), pp. 1503-1506, April 1990. [97] H. V. Sorensen, C. A. Katz, and C. S. Burrus, "Efficient FFT algorithms for DSP processors using tensor product decomposition," in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, (Albuquerque, NM), pp. 1507-1510, April 1990. [98] J. Johnson, R. W. Johnson, D. Rodriguez, and R. Tolimieri, "A methodology for designing, modifying, and implementing Fourier transform algorithms on various architectures," Circuits, Systems and Signal Processing, vol. 9, no. 4, pp. 449-500, 1990. [99] C. Van Loan, Matrix Frameworks for the Fast Fourier Transform. Philadelphia, PA: SIAM, 1992. [100] R. Tolimieri, M. An, and C. Lu, Mathematics of Multidimensional Fourier Transform Algorithms. New York: Springer-Verlag, 1993. [101] J. Granata, M. Conner, and R. Tolimieri, "The tensor product: a mathematical programming language for FFTs," IEEE Signal Processing Magazine, vol. 9, pp. 40-48, January 1992. [102] J. Granata, M. Conner, and R. Tolimieri, "Recursive fast algorithms and the role of the tensor product," IEEE Transactions on Signal Processing, vol. 40, pp. 2921-2930, December 1992.