Try replacing "-f elf64" with "-f macho64" in the x86_64/Makefile as a workaround. Also, you need to make sure you're producing a 64-bit binary while you're compiling. Can't remember how to do that with XCode, but if you don't, the SSE2 code as written will fail. The code uses the added registers that come with the x86_64 architecture to prevent reading/writing to memory.
No dice. I ran the configure with the arch x86_64 command like this:
CFLAGS="-O3 -Wall -msse2 -arch x86_64" ./configure
then edited the x86_64/Makefile as you suggested (elf -> macho). I still get the similar error shown here:
make all-recursive
Making all in compat
Making all in jansson
make[3]: Nothing to be done for `all'.
make[3]: Nothing to be done for `all-am'.
Making all in x86_64
make[2]: Nothing to be done for `all'.
gcc -DHAS_YASM -O3 -Wall -msse2 -arch x86_64 -pthread -o minerd cpu-miner.o util.o sha256_generic.o sha256_4way.o sha256_via.o sha256_cryptopp.o sha256_sse2_amd64.o -lcurl compat/jansson/libjansson.a -lpthread x86_64/libx8664.a
ld: warning: in x86_64/libx8664.a, file was built for unsupported file format which is not the architecture being linked (x86_64)
Undefined symbols:
"_CalcSha256_x64", referenced from:
_scanhash_sse2_64 in sha256_sse2_amd64.o
_scanhash_sse2_64 in sha256_sse2_amd64.o
ld: symbol(s) not found
collect2: ld returned 1 exit status
make[2]: *** [minerd] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2
Seems like something, perhaps gcc (Version: i686-apple-darwin10-gcc-4.2.1 (GCC) 4.2.1 (Apple Inc. build 5664)) , doesn't like the file x86_64/libx8664.a? That or I am not forcing a proper 64bit build for the confused Mac compiler.
For interest, I installed Debian on a virtual machine on my MacBook Pro and compiled cpuminer with SSE2_64 there with no problem. In fact, it runs quite well in the VM - over 3600 hashes/core/sec on the mobile i7, and over 3000 hashes/core/sec on the older Xeons in the MacPro.