It is currently Wed Oct 17, 2018 7:30 pm

All times are UTC - 7 hours





Post new topic Reply to topic  [ 43 posts ]  Go to page Previous  1, 2, 3
Author Message
PostPosted: Fri Sep 14, 2018 5:15 pm 
Offline
User avatar

Joined: Sun Jan 22, 2012 12:03 pm
Posts: 6886
Location: Canada
I only suggested it as a slightly better alternative to being able to accidentally forget to clean. ;P Not a serious suggestion, obviously using real dependencies is better.

gauauu wrote:
But in small NES stuff? Yeah, it's what I do. I can live with an extra 3 seconds of compiling every time I change a header.

Yeah, I considered Make when I started my larger project, and decided to think about it again later if compilation ever got slow. It never did, the full build is still like 1 second. (This is even with like 400k of .byte data in there.)

It doesn't use the CC65 C compiler though, not sure if that is significantly slower? (I've never found it to have noticeable compile times when I have used it.)

Only time I've ever had significant compile time with CC65 was when Movax12 made that Super Mario Bros. disassembly that was composed almost entirely of complicated "high level" macros.


Top
 Profile  
 
PostPosted: Fri Sep 14, 2018 6:54 pm 
Offline
User avatar

Joined: Sat Jan 09, 2016 9:21 pm
Posts: 471
Location: Central Illinois, USA
rainwarrior wrote:
I only
Yeah, I considered Make when I started my larger project, and decided to think about it again later if compilation ever got slow. It never did, the full build is still like 1 second. (This is even with like 400k of .byte data in there.)


My cc65 builds can be a few (2-4) seconds for a full rebuild, particularly if has to rebuild generated assets. Fast enough that I don't care enough to properly set up something to manage the header dependency graph.

What really kills you is you have a bunch of rom data as C files. In my GBA game, each room map generated a C source file of data. Compiling all those could take a couple minutes.

_________________
My games: http://www.bitethechili.com


Top
 Profile  
 
PostPosted: Sat Sep 15, 2018 1:53 am 
Offline
User avatar

Joined: Mon Jan 03, 2005 10:36 am
Posts: 3132
Location: Tampere, Finland
I set up my asset compilation tools (mostly written in Python) to recompile when the source assets (like levels made in Tiled) change. There the compilation times start to stack up enough so that it starts to be a good idea to compile only what's needed.

I threw up a quick example of the cc65 CMake toolchain at https://github.com/fo-fo/cc65-toolchain-example

_________________
Download STREEMERZ for NES from fauxgame.com! — Some other stuff I've done: fo.aspekt.fi


Top
 Profile  
 
PostPosted: Sat Sep 15, 2018 9:52 am 
Offline
User avatar

Joined: Tue Jun 24, 2008 8:38 pm
Posts: 2004
Location: Fukuoka, Japan
@Jarhmander

I just did a quick test and the d files are created but the makefile doesn't react yet to the deps. Right now it is 2h in the morning so it may be the reason that I cannot find the cause ^^;; I think it may be that my $(DEPS) variable may not contain any value and when it does -include $(DEPS) fails to react. Once I find why I think it should work.

Does the location on -include makes a difference? I remember some example in GNU that was putting it more at the top but I don't think that's the reason.

I guess I should try tomorrow after sleeping.

edit:

Need to find at the least the value of DEPS. There was an error, which caused it to be empty but now it contains the list of d files but even if I modify a .h file make doesn't react on it. I tried with -include or include and no difference. This is the only thing left, it's almost working.


Top
 Profile  
 
PostPosted: Sun Sep 16, 2018 6:45 am 
Offline
Formerly ~J-@D!~
User avatar

Joined: Sun Mar 12, 2006 12:36 am
Posts: 472
Location: Rive nord de Montréal
Banshaku wrote:
@Jarhmander

I just did a quick test and the d files are created but the makefile doesn't react yet to the deps. Right now it is 2h in the morning so it may be the reason that I cannot find the cause ^^;; I think it may be that my $(DEPS) variable may not contain any value and when it does -include $(DEPS) fails to react. Once I find why I think it should work.

Does the location on -include makes a difference? I remember some example in GNU that was putting it more at the top but I don't think that's the reason.

I guess I should try tomorrow after sleeping.

edit:

Need to find at the least the value of DEPS. There was an error, which caused it to be empty but now it contains the list of d files but even if I modify a .h file make doesn't react on it. I tried with -include or include and no difference. This is the only thing left, it's almost working.

Maybe the easiest way to diagnose this is to try my repo. It's small and uncomplicated. These makefiles works so if it doesn't when you try it, maybe it's related to the cc65 suite. I remember that there was an issue related to --create-dep in older versions of cc65, though I don't remember what was the problem exactly. If it does work however, it's certainly in how you wrote it.

Attach your modified makefile here and I'll look at how you did it. I cannot test it now because I'm on my puny Pinebook*, maybe this night (EDT).

* It's a nice machine, but certainly not a workhorse. And considering the fact that cc65 is not in this Ubuntu's repo, well I can build it, but only if I have time on my hands. Or cross-compile it, but it's a bit of a pain in its own...

_________________
((λ (x) (x x)) (λ (x) (x x)))


Top
 Profile  
 
PostPosted: Sun Sep 16, 2018 7:40 am 
Offline
User avatar

Joined: Tue Jun 24, 2008 8:38 pm
Posts: 2004
Location: Fukuoka, Japan
I will attach the latest version of my make file. As you will see, not much changed compared to the one I pasted in a few messages above.

The only difference is that I use deps for C files so I do not know if cc65 have issue, like you mentioned. I'm using the latest version, 2.17 unless mistaken.


Attachments:
Makefile_20180916.zip [2.35 KiB]
Downloaded 30 times
Top
 Profile  
 
PostPosted: Mon Sep 17, 2018 8:58 pm 
Offline
Formerly ~J-@D!~
User avatar

Joined: Sun Mar 12, 2006 12:36 am
Posts: 472
Location: Rive nord de Montréal
I have found the problem. It's quite simple actually when we see the Makefile in action.

gcc and clang both compile "c" files to "o" directly; cc65 cannot. Instead, it outputs an assembly file which can be assembled. Because of this, the generated dep file looks like this:
Code:
build/src/test.s:   src/test.c src/test.h

src/test.c src/test.h:

That's logical: it cannot know yet of the real output file, so dependency is expressed from the point of view of its real output, the "s" file. And here lies the entirety of the problem. Your Makefile does this with "c" files:
Code:
### Main target files to compile ###
$(C_OBJ_FILES): $(BUILD_DIR)/%.o : %.c
   @mkdir -p $(@D)
   $(CC) $(CCFLAGS) $(CREATE_DEP) $< -o $(@:.o=.s)
   $(AS) $(@:.o=.s) $(ASFLAGS) -l $(@:.o=)_listing.txt -o $@
Here, the rules says that for every C_OBJ_FILES, each "o" files depends on "c" files. But nothing tells make about the "s" files generated along the way. Well, the dep file mentions some dependencies, yes, but make doesn't know that "o" files are generated from "s" files from "c" files; the recipe assembles "s" files behind his back.

There are two valid solutions: split this rules into two rules, one doing "c" → "s" and the other from "s" → "o", OR use cl65.
Yes, cl65, with the -c flag, will compile, assemble but not link source files. The dep file will nonetheless reflect the expected dependencies: now things depends on the "o" file, which the Makefile knows about. This is the solution I prefer, because it acts more like the other compilers, and it simplify the Makefile a (tiny) bit. There are the only things to watch for with cl65:
  • The order of options vs argument is important; it's better to put the source file at the end of the command line, because what's after it will not be taken into account;
  • The default target is C64, use -t none to counter this.
  • Most options are understood by cl65 directly, but if for some reason you need a specific compiler/assembler option that cl65 doesn't know about, then, and only then use the -Wc/-Wa option to pass that option through cl65.

So here's the diff which is quite small, and attached is the modified Makefile.
Code:
diff --git a/Makefile b/Makefile
index 875a24b..2ffd942 100644
--- a/Makefile
+++ b/Makefile
@@ -44,8 +44,8 @@ TARGET = $(APP_NAME).nes
 
 ##################
 # CC65 related
-AS := ca65
-CC := cc65
+AS := cl65
+CC := cl65
 LD := ld65
 AR := ar65
 
@@ -95,8 +95,8 @@ FT_DEFINES += -D FT_SFX_STREAMS=4    # Set all 4 channels for SFX
 
 ###################
 # Compiler/linker flags
-ASFLAGS = -g -I $(SRC_DIR) $(INCLUDE_LIBS_FLAGS) $(INCLUDE_DATA_FLAGS) $(FT_DEFINES)
-CCFLAGS = --add-source -Oi -Cl -g $(INCLUDE_LIBS_FLAGS)
+ASFLAGS = -c -t none -g -I $(SRC_DIR) $(INCLUDE_LIBS_FLAGS) $(INCLUDE_DATA_FLAGS) $(FT_DEFINES)
+CCFLAGS = -c -t none --add-source -Oi -Cl -g $(INCLUDE_LIBS_FLAGS)
 LDFLAGS = -Ln $(BUILD_DIR)/$(APP_NAME)_labels.txt -m $(BUILD_DIR)/$(APP_NAME)_map.txt --dbgfile $(APP_NAME).dbg -vm
 INCLUDE_DATA_FLAGS := $(addprefix -I ,$(DATA_DIR_LIST)) $(addprefix --bin-include-dir ,$(DATA_DIR_LIST)) $(addprefix --bin-include-dir ,$(SRC_DIR_LIST))
 INCLUDE_LIBS_FLAGS := $(addprefix -I ,$(SRC_DIR_LIST)) $(addprefix -I ,$(LIB_DIR_LIST))
@@ -128,12 +128,11 @@ print-%  : ; @echo $* = $($*)
 ### Main target files to compile ###
 $(C_OBJ_FILES): $(BUILD_DIR)/%.o : %.c
        @mkdir -p $(@D)
-       $(CC) $(CCFLAGS) $(CREATE_DEP) $< -o $(@:.o=.s)
-       $(AS) $(@:.o=.s) $(ASFLAGS -l $(@:.o=)_listing.txt -o $@
+       $(CC) $(CCFLAGS) $(CREATE_DEP) -l $(@:.o=)_listing.txt -o $@ $<
 
 $(S_OBJ_FILES): $(BUILD_DIR)/%.o : %.s
        @mkdir -p $(@D)
-       $(AS) $< $(ASFLAGS) $(CREATE_DEP) -l $(@:.o=)_listing.txt -o $@
+       $(AS) $(ASFLAGS) $(CREATE_DEP) -l $(@:.o=)_listing.txt -o $@ $<
 
 ##########################
 ### Build runtime only ###


Attachments:
Makefile.zip [2.38 KiB]
Downloaded 30 times

_________________
((λ (x) (x x)) (λ (x) (x x)))
Top
 Profile  
 
PostPosted: Mon Sep 17, 2018 11:25 pm 
Offline
User avatar

Joined: Tue Jun 24, 2008 8:38 pm
Posts: 2004
Location: Fukuoka, Japan
I see. I don't remember why I didn't use cl65 directly, maybe because I wanted to keep the .s file for some reason but today that wouldn't be an issue. Thank you for going that far to find the cause!

I did a quick compile and it's almost working. For some reason some include paths are not working anymore for s file. Once it compile and deps works as expected, I will let you know. The extension is .inc but that shouldn't be an issue. Must be a little something, will figure it out soon.


Top
 Profile  
 
PostPosted: Wed Sep 19, 2018 8:05 pm 
Offline
Formerly ~J-@D!~
User avatar

Joined: Sun Mar 12, 2006 12:36 am
Posts: 472
Location: Rive nord de Montréal
Have you found what's causing your issue? I'm not sure how the modifications break compilation of "s" files; if anything, I changed how the "c" files are compiled, so it would expect that if I broke something, that must be it. Assembly files are compiled through cl65, and I only made a slight reordering of the command-line flags.

That might be a bit extreme, but if you don't find what's causing this, you can attach the console output of a clean compile (e.g. with make | tee output.log), one with the old Makefile and the other with the new. I might figure out something from those outputs alone.

_________________
((λ (x) (x x)) (λ (x) (x x)))


Top
 Profile  
 
PostPosted: Wed Sep 19, 2018 9:03 pm 
Offline
User avatar

Joined: Tue Jun 24, 2008 8:38 pm
Posts: 2004
Location: Fukuoka, Japan
I didn't work on it recently but I do not think what you updated is the cause. From the way it's done, everything seems fine. My guess is (but I need to confirm by testing) maybe the include list is too long and cl65 could be failing on that because of the error received. Since the workflow didn't change except to switch from as/cc to cl, this seems to be the current logical explanation. It could be a bug in the parsing of -i but until I can prove it, this is just speculation.

The way my project work is, instead of putting relative path in every file, I just list all folder in src and data and include them on every item compiled. This avoid issue with relative path when using files from other projects and allow to move file without worrying about the include. The compromise to be done is that all include file name must be unique, if not then you have to write the real path in that case.

Except for the include path, I don't think what you wrote is the cause. Maybe what I will do is test this makeFile with a sample with less include and see if it does work. I updated the sfx test sample that glutock gave me with my makeFile so it would be the perfect test sample for that since only a few folder exist. If it works then test an s file with inc files and if it does work then maybe what I'm assuming is the cause is right.

I will see if I can test since it should be quite fast with a small sample.


Top
 Profile  
 
PostPosted: Wed Sep 19, 2018 9:41 pm 
Offline
User avatar

Joined: Tue Jun 24, 2008 8:38 pm
Posts: 2004
Location: Fukuoka, Japan
Sorry for the double post. Here's the error log. I provided a sample so it would be easier to figure out.

Code:
cl65 -c -t none --add-source -Oi -Cl -g -I src -I libs -I libs/famitone/ -I libs/ft_drv/ -I libs/neslib/ --create-dep build/src/example.d -l build/src/example_listing.txt -o build/src/example.o src/example.c
cl65 -c -t none -g -I src -I src -I libs -I libs/famitone/ -I libs/ft_drv/ -I libs/neslib/ -I data --bin-include-dir data --bin-include-dir src -D FT_DPCM_ENABLE=1     -D FT_SFX_ENABLE=1      -D FT_THREAD=1          -D FT_PAL_SUPPORT=1     -D FT_NTSC_SUPPORT=1    -D FT_SFX_STREAMS=4     --create-dep build/src/crt0.d -l build/src/crt0_listing.txt -o build/src/crt0.o src/crt0.s
src/crt0.s(240): Error: Constant expression expected
src/crt0.s(257): Error: Cannot open include file `ft_drv/driver.s': No such file or directory
src/crt0.s(258): Error: Cannot open include file `neslib.s': No such file or directory
src/crt0.s(272): Error: Constant expression expected
src/crt0.s(279): Error: Constant expression expected
Makefile:139: recipe for target 'build/src/crt0.o' failed
make: *** [build/src/crt0.o] Error 1



my guess was wrong, it only fails with .s file actually so it cannot be the length of include paths. I will continue to test with that sample since it is easier to figure out from a simple set.

Small notes:
- MakeFile.old is the makeFile without update
- I added AS2 in new one since I wanted to do a quick test and wasn't sure the proper parameters to add for building lib. I will figure out later

edit:

The error was similar (I think) with my code, the constant expression I'm not sure but I will check during my lunch but I think it is almost the same. I will retest with my code later and add extra comment to this post and fix the sample is there is something wrong in it.

edit3:

removed edit2 since the doc as an example later for .s file. I was just mislead by how the top explanation was written. Continue to check possible causes. For the "Constant expression expected", it's like it doesn't recognize the -D parameters somehow. It was not an issue with as65.

edit4:

For c files, deps is working fine and was able to see it in action for h file, which is nice. Now I just need to figure out what it doesn't like with the parameters for that s file since executing that same command (minus -c -t none) with ca65 works fine. hmmm...

edit5:

Hopefully my last edit. My current solution to the problem is:

- CC uses cl65
- AS uses ca65

If used that way, both recognise when files are updated and compile properly! I don't know why cl65 fails with parameters for .s file since the same parameters works with ca65. It could be a bug in 2.17, who knows. For now, unless you think there is an issue to do it that way, I'm more than happy with the result.

Thanks!!!!


Attachments:
shiru-ft-sfx-issue2_makefile_deps_tests.zip [313.77 KiB]
Downloaded 28 times
Top
 Profile  
 
PostPosted: Wed Sep 19, 2018 11:14 pm 
Offline
Formerly ~J-@D!~
User avatar

Joined: Sun Mar 12, 2006 12:36 am
Posts: 472
Location: Rive nord de Montréal
Hmm, this one is quite funny. From cl65's source code, -D and -I are only handed to cc65, not ca65. In the resulting ASFLAGS, substituting --asm-define for -D and --asm-include-dir for -I will make it work, though I don't know if it's really an improvement over just using ca65 at this point?

gcc's behavior is different:
Code:
% gcc -c -x assembler-with-cpp -DFOO=42 -o test.o - <<.
    .global Foo

    Foo = FOO
.
% nm test.o
000000000000002a A Foo


I don't know if it deserves a bug report for cl65?

_________________
((λ (x) (x x)) (λ (x) (x x)))


Top
 Profile  
 
PostPosted: Wed Sep 19, 2018 11:43 pm 
Offline
User avatar

Joined: Tue Jun 24, 2008 8:38 pm
Posts: 2004
Location: Fukuoka, Japan
I see. So I was right about what I was suspecting but I usually try to think that "it must" be a user error so don't blame the tool until you're 100% sure. I make so many error that I usually blame myself first over everything else :lol:

Jarhmander wrote:
I don't know if it deserves a bug report for cl65?


Since the doc doesn't mention that -I and -D doesn't work for ca65 then yes, I think it would be a good idea to create one.

If you read the doc from the top, the explanation gave me the false impression that the main target was for C after reading that part:

Quote:
The cl65 compile and link utility may be used to convert, compile, assemble and link files. While the separate tools do just one step, cl65 knows how to build object files from C files (by calling the compiler, then the assembler) and other things.


But it could be that sometime I try to read "too much" between the lines and it didn't mean that ^^;;; After reading the doc further, they give some example with file extention so this is when I was sure that it was for both.

Where should the bug report be written, on github?

edit:

Oh, and thanks again for going that far to confirm, I really appreciate it. It now works so that will save a lot of time!


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 43 posts ]  Go to page Previous  1, 2, 3

All times are UTC - 7 hours


Who is online

Users browsing this forum: Bavi_H, strat and 4 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group