Please note:The SCons wiki is in read-only mode due to ongoing spam/DoS issues. Also, new account creation is currently disabled. We are looking into alternative wiki hosts.
   1 17:18:19 *	stevenknight (n=stevenkn@nat/google/x-ff6fd1a26f0686e6) has joined #scons
   2 17:25:16 *	Jason_ ( has joined #scons
   3 17:29:26 *	garyo-home (n=chatzill@ has joined #scons
   4 17:31:26 *	GregoryNoel is here
   5 17:31:38 <GregoryNoel>	Hi, guys, are we ready?
   6 17:31:51 <garyo-home>	Hi Greg.  Ready as I'll be. :-/
   7 17:32:23 <GregoryNoel>	I only see limited comments in the spreadsheet; it's gonna be a slow night.
   8 17:32:53 <garyo-home>	yes, sorry.  Just doing some now.
   9 17:33:02 <stevenknight>	hey
  10 17:33:07 <stevenknight>	me too
  11 17:34:15 <GregoryNoel>	1945?
  12 17:34:42 <garyo-home>	steven to research & pick one option.
  13 17:34:47 <garyo-home>	w/ Ludwig.
  14 17:34:57 <garyo-home>	IMHO.
  15 17:35:25 <GregoryNoel>	I'll go with that, at least for now.  We need to come up with something better in the long run.
  16 17:36:17 <garyo-home>	Not sure about your comment re: rewriting implicit dependency logic, but won't go there now.
  17 17:36:44 <stevenknight>	agree w/what you guys are saying
  18 17:37:17 <GregoryNoel>	I think we can do more than what we are now by caching negatives, but it's not the time to redesign it.
  19 17:37:36 <garyo-home>	right.
  20 17:38:00 <GregoryNoel>	so research, Steven + Ludwig?
  21 17:38:11 <garyo-home>	good.
  22 17:38:19 <GregoryNoel>	And once the decision is reached, Ludwig to implement?
  23 17:38:28 <stevenknight>	done
  24 17:38:31 <GregoryNoel>	done
  25 17:38:35 <GregoryNoel>	2258
  26 17:38:36 <garyo-home>	2258: stevenknight to add a hook to get help text, 2.x p4?
  27 17:39:07 <GregoryNoel>	Hmmmm.....
  28 17:40:06 <GregoryNoel>	Yeah, something like that.  His suggestion is awkward, but it should be possible to work out something.
  29 17:40:38 <GregoryNoel>	And for 2.x p4, we don't need to volunteer anyone.
  30 17:40:41 <GregoryNoel>	yet
  31 17:40:48 <garyo-home>	agree.
  32 17:40:52 <stevenknight>	okay, i'm back from the spreadsheet
  33 17:41:04 <stevenknight>	2258:  done
  34 17:41:09 <stevenknight>	2.x p4, future draft pick
  35 17:41:10 <GregoryNoel>	2261
  36 17:41:38 <stevenknight>	it's a google guy, so i'd like a crack at it
  37 17:41:58 <GregoryNoel>	We have another issue looking at this already; one should be closed.
  38 17:42:00 <garyo-home>	Shouldn't he be using the target filename as the target?  (And not rm/mkdir either)
  39 17:41:59 <stevenknight>	if i can't come up with a real case, i'll mark it invalid
  40 17:42:35 <garyo-home>	ok, that's good enough for me.  Steven to find testcase or mark invalid.
  41 17:43:05 <GregoryNoel>	or dup to the other one
  42 17:43:16 <stevenknight>	yeah, i guess i view it similar to unpacking a .tar.gz into a directory
  43 17:43:22 <stevenknight>	i think you should be able to specify the directory
  44 17:43:30 <stevenknight>	and have it do the right thing w.r.t. changes to the source(s)
  45 17:43:41 <stevenknight>	but i'll mark it invalid if I'm just dreaming
  46 17:43:47 <GregoryNoel>	done
  47 17:44:00 <GregoryNoel>	2262
  48 17:44:12 <stevenknight>	i think it should be trivial
  49 17:45:26 <GregoryNoel>	If you think it's trivial, I'll let you have it.
  50 17:45:20 <stevenknight>	2262:  anytime, anyone
  51 17:45:29 <GregoryNoel>	anytime is fine
  52 17:45:39 <garyo-home>	The OP says he'd like "not .py" but I think .py makes the most sense.
  53 17:45:41 <stevenknight>	okay, 2262 anytime stevenknight
  54 17:46:15 <Jason_>	I like the .scons ,myself
  55 17:46:17 <stevenknight>	while we're at it, i'd go for both .py and .scons
  56 17:46:36 <stevenknight>	and draw the line there
  57 17:46:38 <GregoryNoel>	I've also seen .sc used.
  58 17:46:45 <garyo-home>	ok, why not.  Just pick an order, seems fine to me.
  59 17:47:04 <garyo-home>	Greg: I think .sc is too short.
  60 17:47:17 <GregoryNoel>	Sigh.  Seems too complex; I'd still wontfix it as I indicated.
  61 17:47:02 <stevenknight>	isn't .sc for SuperCalc files...?
  62 17:47:05 *	stevenknight shows his age...
  63 17:47:12 <garyo-home>	SuperCalc?!!
  64 17:47:38 <GregoryNoel>	Yeah, I remember it, too.
  65 17:47:46 *	GregoryNoel _really_ shows his age.
  66 17:47:47 <stevenknight>	2262:  anytime, stevenknight
  67 17:47:52 <GregoryNoel>	done
  68 17:48:03 <GregoryNoel>	2264
  69 17:48:27 <garyo-home>	Greg, you looked at this most -- is the order reversal the only thing going on here?
  70 17:48:28 <stevenknight>	2.x p2 gregnoel?
  71 17:49:16 <GregoryNoel>	No, the dependencies aren't reversed, reversing the order of the names on the command line produces different trees.
  72 17:49:43 <garyo-home>	ok, I believe you :-)
  73 17:49:47 <GregoryNoel>	No matter which way you list the names on the command line, it should show the same dependencies.
  74 17:49:57 <garyo-home>	do you mind taking it?
  75 17:50:45 <GregoryNoel>	I know almost nothing about that part of the code, so I'd rather not, but if you push, I'll consider it.
  76 17:51:04 <stevenknight>	2.x p2 can be future draft pick
  77 17:51:16 <stevenknight>	or else put my name on it
  78 17:51:16 <GregoryNoel>	I'll go with that
  79 17:51:27 <stevenknight>	okay, 2.x p2 future draft pick
  80 17:51:28 <stevenknight>	done
  81 17:51:40 <GregoryNoel>	2265
  82 17:51:50 <garyo-home>	ah, yes.
  83 17:51:56 <Jason_>	just so it is known this is not new
  84 17:52:15 <stevenknight>	2265:  1.2 p1 stevenknight
  85 17:52:16 <garyo-home>	Jason_, you mean 2265?
  86 17:52:23 <Jason_>	Yes
  87 17:52:27 <stevenknight>	like I said, I'm just in Taskmaster lately
  88 17:52:28 <Jason_>	I filed it
  89 17:52:30 <garyo-home>	thanks, Steven.
  90 17:52:47 <garyo-home>	Ah, you're *that* Jason -- welcome!
  91 17:52:55 <Jason_>	Thanks
  92 17:53:13 <GregoryNoel>	stevenknight, done
  93 17:53:19 <GregoryNoel>	and thanks
  94 17:53:18 <stevenknight>	Jason_:  glad to have you here
  95 17:53:24 <Jason_>	hope we can talk latter :-)
  96 17:53:38 <stevenknight>	yep
  97 17:53:46 <stevenknight>	i have a few non-issue topics i'd like to discuss as well
  98 17:53:54 <garyo-home>	ok
  99 17:54:12 <GregoryNoel>	2266: I'll let you guys fight it out
 100 17:54:17 <stevenknight>	2266:  1.x p3 stevenknight
 101 17:54:36 <Jason_>	we have seen this .. it depends on the python used
 102 17:54:43 <garyo-home>	2266: not sure why this doesn't work for him.  I do this all the time.  What python is he using?
 103 17:54:58 <garyo-home>	Do NOT use cygwin python.
 104 17:55:15 <Jason_>	That follows what we see here as well
 105 17:55:29 <Jason_>	cygwin python will mess up a windows based build
 106 17:55:37 <garyo-home>	2266: let me take it and suggest that to him.
 107 17:55:43 <GregoryNoel>	done
 108 17:55:54 <GregoryNoel>	2267
 109 17:56:47 <stevenknight>	either 1.2 and defer if investigation shows we can live with it
 110 17:56:56 <GregoryNoel>	This is the new code I put in, but I can't see how it's not initialized.
 111 17:56:57 <stevenknight>	or resesarch with the option of making it a 1.2 blocker
 112 17:58:19 <GregoryNoel>	Seeing no hands, I'll research it.  It's a bad week for that, but I'll try to make time.
 113 17:58:22 <garyo-home>	Greg: possible exception issue?
 114 17:58:27 <GregoryNoel>	garyo-home, huh?
 115 17:58:35 <garyo-home>	never mind.
 116 17:59:15 <GregoryNoel>	so 2267 research me?
 117 17:59:20 <stevenknight>	done
 118 17:59:47 <stevenknight>	GregoryNoel:  let me know if I can help, I've had to debug things like this many times
 119 17:59:53 <GregoryNoel>	thanks
 120 18:00:05 <GregoryNoel>	Before we go on, can I insert a topic?
 121 18:00:17 <garyo-home>	sure
 122 18:00:34 <stevenknight>	sure, let's do GN's first, then Jason_, then me
 123 18:00:30 <GregoryNoel>	I'd like to change the agenda slightly.  I'd like to add a permanent item to discuss the current status of the release schedule and how we're doing on it.
 124 18:00:30 <GregoryNoel>	Discussion points would be what's expected in the next release(s) (whether checkpoint, candidate, or public), adjustments to the schedule, whether we should do any retriage for current or future releases, things like that.
 125 18:00:30 <GregoryNoel>	I'd put it between the current issues and the backlog.  (Aside: we're almost to the home stretch for finishing off the backlog (hopefully we'll kill off 2005 this time or next time); after that, the time now spent on backlog will be taken up by bickering about retriage of issues, so we should discuss the release schedule strategy before the tactics.  This positioning puts it in the right place for the future.)
 126 18:01:13 <stevenknight>	+1 to discussing release schedule regularly
 127 18:01:24 <garyo-home>	greg: +1 to your agenda change.  I always review the schedule before the meeting anyway.
 128 18:02:16 <GregoryNoel>	OK, I'll do it.  Since the slot would be now, should we discuss the schedule now?
 129 18:02:21 <stevenknight>	yes
 130 18:02:23 <garyo-home>	yes.
 131 18:02:31 <GregoryNoel>	Schedule:  1.2 was due out 24 November.  Needless to say, we didn't make it.  And issues 2265 and 2267 could be a blockers; we may need to slip 1.2 until that's resolved.
 132 18:02:31 <GregoryNoel>	We were supposed to have a checkpoint out 1 December.  We didn't make that, either.
 133 18:02:31 <GregoryNoel>	Assuming we get the release out this week, we have some decisions to make.  Should we slide the rest of the schedule by a week, so the next checkpoint is 8 December?  Cancel the 1 December checkpoint and keep the current schedule, with the next checkpoint 15 December?  Or something else?
 134 18:02:31 <GregoryNoel>	I lean toward dropping the 1 December checkpoint, but I could be easily persuaded differently.
 135 18:02:59 <stevenknight>	i put out a release candidate 11/25?
 136 18:03:33 <GregoryNoel>	I think that's about right
 137 18:03:28 <garyo-home>	I have three 1.2 tickets on my plate.  None serious except maybe #2048 which I'm working on but fails in weird ways
 138 18:04:42 <stevenknight>	oh, wait, I see--the release schedule is 12/1 checkpoint that was supposed to be *after* 1.2 was out
 139 18:04:45 <stevenknight>	and i'm late with 1.2
 140 18:04:51 <stevenknight>	right?
 141 18:05:10 <garyo-home>	I think so.  I'd like to get 1.2 out soon then put vs_revamp in for next checkpoint after that.
 142 18:05:40 <GregoryNoel>	stevenknight, yes
 143 18:05:37 <stevenknight>	right, and I actually have batched Visual C/C++ compilation working on a private branch that I'd like to go in
 144 18:05:56 <garyo-home>	wow!
 145 18:05:57 <Jason_>	I would like to add stuff to the revamp
 146 18:06:12 <garyo-home>	Jason: good.  You have an version of it too I think.
 147 18:06:12 <stevenknight>	okay, so let's review 1.2 issues and push anything that's not a potential/likely blocker in 1.3
 148 18:06:18 <Jason_>	We have some work not finished that will improve it and make it faster
 149 18:06:38 <garyo-home>	OK.  Let's get it into the trunk, then review & apply your changes
 150 18:06:46 <Jason_>	yep... still working on that part
 151 18:06:48 <garyo-home>	ok?
 152 18:07:02 <Jason_>	plus we have early support for vs 2010 and intel 11
 153 18:07:02 <stevenknight>	garyo_home:  "it" into the trunk:  "it" == vs_revamp?
 154 18:07:14 <stevenknight>	Jason_:  sweet
 155 18:07:31 <garyo-home>	steven: yes, it==vs_revamp.
 156 18:07:36 <Jason_>	I have "people" that know "people"
 157 18:07:38 <stevenknight>	okay, agreed
 158 18:08:01 <Jason_>	Well my fixes to the revamp.. are a bit different from the work David did
 159 18:08:02 <garyo-home>	There are 40 open 1.2 issues.
 160 18:08:05 <GregoryNoel>	Almost 40 issues scheduled for 1.2 still
 161 18:08:13 <GregoryNoel>	garyo-home, jinx
 162 18:08:47 <garyo-home>	Jason_: send a summary of your ideas & differences to the list, we'll discuss it there.
 163 18:08:55 <Jason_>	ok
 164 18:09:13 <garyo-home>	It's a bigger forum, David will be present, etc.
 165 18:09:16 <stevenknight>	a lot of the 40 issues are readily rollable to 1.3
 166 18:09:19 <stevenknight>	doc issues, e.g.
 167 18:09:57 <GregoryNoel>	Mine are all doc and testing (I haven't figured out how to test Action._subproc yet)
 168 18:10:05 <garyo-home>	15 P2s, the rest P3 & P4.
 169 18:10:49 <stevenknight>	oh, and 40 doesn't count the two potential blockers we discussed tonight
 170 18:11:15 <stevenknight>	quick, quick glance suggests all P3 & P4 are automatically deferrable
 171 18:12:07 <garyo-home>	agreed.
 172 18:12:09 <stevenknight>	and I think the cournape P2 issues are because we once thought vs_revamp was going to make it into 1.2
 173 18:12:54 <garyo-home>	Is Benoit around?  Any likelihood of his 4 issues getting done?
 174 18:13:06 <stevenknight>	slim, he's been AWOL for a while
 175 18:13:21 <garyo-home>	thought so.
 176 18:13:31 <stevenknight>	worth considering reassigning his issues so they don't languish
 177 18:14:04 <garyo-home>	Yes.  I suggest moving to 1.3 for now, then reassigning.
 178 18:15:02 <GregoryNoel>	garyo-home, agree
 179 18:15:13 <stevenknight>	okay, so it sounds like the only two issues that don't go to 1.3 are the two new potential blockers
 180 18:15:32 <GregoryNoel>	I can do that with a mass move.
 181 18:15:41 <stevenknight>	GregoryNoel:  thanks
 182 18:15:33 <stevenknight>	we get patches for those (or research suggests deferring to 1.3) and then I cut another 1.2 candidate
 183 18:16:02 <garyo-home>	What's up w/ 2249 though?
 184 18:16:44 <stevenknight>	dunno...
 185 18:16:47 <garyo-home>	(Not that it should be in 1.2 anyway, but how'd it get there?)
 186 18:17:07 <stevenknight>	vs_revamp, we thought it would be in 1.2
 187 18:17:08 <garyo-home>	It can move to 1.3 too since it's vs_revamp related.
 188 18:17:59 <garyo-home>	ok, so given all that, what's the proposed 1.2 release date?
 189 18:18:10 <GregoryNoel>	ASAP?
 190 18:18:13 <stevenknight>	stake in the ground:  12 December, a week from Friday
 191 18:18:20 <stevenknight>	candidate 5 December, two days
 192 18:18:32 <stevenknight>	for the two potential blockers
 193 18:18:41 <GregoryNoel>	works; I'll update the roadmap
 194 18:18:44 <garyo-home>	ok, so I have some time to sneak in some little things before the candidate, then we freeze, right?
 195 18:19:00 <stevenknight>	yep
 196 18:19:04 <garyo-home>	good.
 197 18:19:20 <stevenknight>	okay, shall we move on to Jason_'s topic?
 198 18:19:37 <garyo-home>	yes.
 199 18:19:57 <garyo-home>	I looked a little at the doc for Parts; it is quite serious!
 200 18:20:22 <Jason_>	Well we have some new tool coming out... it has to be a bit serious
 201 18:20:41 <garyo-home>	Jason, I wonder if some of it couldn't be done with SCons Tools instead, but basically it looks useful.
 202 18:20:49 <garyo-home>	What's the new Intel tool coming out?
 203 18:21:05 <Jason_>	well I have a lot to say on all of this
 204 18:21:21 <Jason_>	Idea the doc i sent is about extending Scons...
 205 18:21:31 <garyo-home>	The basic idea is a way to make a scalable composition of libs etc., right?
 206 18:21:32 <Jason_>	here i think tools needs many improvements
 207 18:22:05 <Jason_>	well yes... but also about making large project sain to deal with
 208 18:22:33 <garyo-home>	Right, by imposing certain conventions that make it subdividable nicely.
 209 18:22:36 <Jason_>	having a unified build system is very important
 210 18:22:55 <Jason_>	but also trying to be flexible
 211 18:23:07 <stevenknight>	sorry, I haven't had time to look before now
 212 18:23:09 <Jason_>	it is a hard balance...
 213 18:23:12 <garyo-home>	Of course; lots of people have large unified build systems with SCons.  Your way is one way of formalizing a structure.
 214 18:23:18 <stevenknight>	is the doc the .7z file you attached?
 215 18:23:29 <garyo-home>	Steven: yes, it's in there.
 216 18:23:37 <Jason_>	yes.. the problem with the team i work with is that we are all over the place
 217 18:23:48 <Jason_>	having a "make system" does not work well
 218 18:23:56 <Jason_>	yes
 219 18:24:16 <stevenknight>	agh, i don't have 7zip on this system
 220 18:24:24 <Jason_>	it is in word 2007 format.. I can give you an update in something else like html
 221 18:25:25 <Jason_>	so the first question for me is how can we best stare this code
 222 18:25:54 <Jason_>	My boss would like this to be part of SCons instead of it own project...
 223 18:26:27 <Jason_>	however it might be best to have it as a seperate project with heavy SCons involment
 224 18:27:11 <garyo-home>	I've long thought it would be good for SCons to have a contrib/ subtree, so that might be one way to go.  The downside is you're tied to SCons release schedule and other stuff though.  Separate is probably easier.
 225 18:27:36 <Jason_>	well we have our own svn branch here at Intel
 226 18:27:48 <Jason_>	ideally we would try to move to a public one
 227 18:28:15 <Jason_>	but we have some stuff in a subdir call "pieces" that i cannot ship
 228 18:28:33 <garyo-home>	Not sure what the best way to host a project like that is, but there are many places to look.
 229 18:28:44 <Jason_>	as it is internal add on that have knowleage of internal Intel networks
 230 18:28:59 <garyo-home>	You'll have to make it work "out of the box" from the public repo, of course.
 231 18:29:10 <Jason_>	yep that is the idea
 232 18:29:22 <Jason_>	the sample in the code i gave you should work out of the box
 233 18:29:33 <garyo-home>	I think everyone will be happier if it is separate from scons but closely related, at least for now.
 234 18:29:37 <Jason_>	the full sample will need svn to be installed :-)
 235 18:30:09 <garyo-home>	But we should spend some time at least reading the doc to get an idea of the scope and goals of the project.
 236 18:30:17 <Jason_>	If this agreed, I will start the process to make a tigris Parts project
 237 18:30:39 <garyo-home>	How about calling it SConsParts or something to show its relation to SCons?
 238 18:30:58 <Jason_>	that is fine with me
 239 18:31:12 <Jason_>	any other votes?
 240 18:31:58 <garyo-home>	Let's talk about the tech details and direction more at the next bug party meeting in 2 weeks.
 241 18:32:11 <stevenknight>	sorry, still trying to open things up
 242 18:32:28 <stevenknight>	got the .7z open, but i don't have Office on the system I switched to...
 243 18:32:28 <Jason_>	in 2 week I will be in florida *not* working
 244 18:32:31 <garyo-home>	btw, is this going to appear in a public tool from Intel?
 245 18:32:35 <Jason_>	so after that is fine
 246 18:33:01 <garyo-home>	Jason: ok, whenever you are around.  But give us a week or so to look it over.
 247 18:33:10 <Jason_>	sure
 248 18:33:14 <garyo-home>	Esp. given 1.2 release schedule :-)
 249 18:33:29 <stevenknight>	Jason_:  there's an early effort here at Google to do something that sounds similar to Parts
 250 18:33:38 <stevenknight>	but it looks like you're much farther along
 251 18:33:46 <stevenknight>	when I look, I
 252 18:33:47 <Jason_>	so the parts stuff will be public.. MIT under Intel with the intent to be move as much as possible to SCons
 253 18:34:02 <Jason_>	you work at Google
 254 18:34:14 <stevenknight>	I'm going to look for areas of possible cooperation
 255 18:34:27 <stevenknight>	yes, almost 11 months now
 256 18:34:38 <Jason_>	oh .. I thought it was VMware
 257 18:34:45 <stevenknight>	that was before, and on contract
 258 18:34:56 <Jason_>	well you should contact me... INtel and google have a lot going on
 259 18:35:18 <stevenknight>	the problem you're addressing is what we're running into too
 260 18:35:46 <Jason_>	so are many projects at Intel ....
 261 18:35:56 <stevenknight>	right, SCons is pretty good low-level plumbing
 262 18:36:08 <Jason_>	very powerful stuff however
 263 18:36:15 <stevenknight>	but there's no consistency for higher layer plug-and-play between components
 264 18:36:35 <stevenknight>	esp. in the open source world it all depends on how the person wrote the SCons config
 265 18:36:37 <Jason_>	I think as i under scons better and you parts.. I think there is a lot of value here to improve on
 266 18:36:47 <stevenknight>	agreed
 267 18:37:10 <Jason_>	the plug and play in very important
 268 18:37:30 <stevenknight>	right, it's the real value add of Autotools (IMHO)
 269 18:37:31 <Jason_>	when I was at a start up before intel.. one person controled the make scripts
 270 18:37:58 <stevenknight>	being able to approach any package and know how to at least build it consistently
 271 18:38:12 <Jason_>	in there larger cross geo.. developer fight over details and perfection
 272 18:38:14 <stevenknight>	sounds like Parts goes even further though
 273 18:38:23 <Jason_>	so everything breaks .
 274 18:38:26 <stevenknight>	right
 275 18:38:32 <stevenknight>	so following up on what Gary said
 276 18:38:42 <Jason_>	cross geo poltics i have found can make it hard to get simple stuff done
 277 18:38:56 <stevenknight>	I'm going to take a closer look after 1.2 is out
 278 18:39:03 <Jason_>	ok
 279 18:39:24 <stevenknight>	and perhaps touch base with you off line to compare first impressions
 280 18:39:26 <Jason_>	I will send out friday an update to the code and document ( small tweaks and fixes)
 281 18:39:33 <stevenknight>	thanks
 282 18:39:41 <Jason_>	so FYI
 283 18:39:41 <stevenknight>	if you can convert doc to some non-Word format that'd help me
 284 18:39:51 <garyo-home>	same here
 285 18:39:49 <Jason_>	WW 51 i am out of town
 286 18:40:05 <garyo-home>	What is "WW 51"?
 287 18:40:08 <Jason_>	 I will be around other wise and will keep track of the e-mails
 288 18:40:23 <Jason_>	week of Dec 15
 289 18:40:34 <garyo-home>	ah yes.
 290 18:40:43 <stevenknight>	okay, i'll look for your update and we'll go from there
 291 18:40:50 <Jason_>	ok
 292 18:40:57 <Jason_>	thank you for your time
 293 18:41:12 <garyo-home>	ok, Steven I think it's your turn now!
 294 18:41:17 <stevenknight>	ok
 295 18:41:21 <garyo-home>	Jason_: yr welcome.
 296 18:41:40 <stevenknight>	one is the Visual C/C++ batch builder change I mentioned
 297 18:41:58 <stevenknight>	took a while, but it's close to solid
 298 18:42:09 <stevenknight>	some unit tests need attention
 299 18:42:27 <garyo-home>	Impressive, can't wait to try it.  We have a lot of vc++ compiled into a few big libs.
 300 18:42:54 <stevenknight>	i'm going to try to get it in right away after 1.2 is out so it gets soak time
 301 18:43:16 <stevenknight>	it has some changes to the Taskmaster that aren't terribly extensive
 302 18:43:23 <stevenknight>	but may have unintended side effects
 303 18:43:31 <GregoryNoel>	(BRB.  My wife just arrived with food and I've got to eat it while it's hot.)
 304 18:43:56 <garyo-home>	Do users have to do anything or does it just work?
 305 18:43:59 <stevenknight>	darn, bad timing, I was going to ask Greg something about Taskmaster NG
 306 18:44:03 <Jason_>	If you need someone to help test the taskmaster on a large project we are happy to help
 307 18:44:24 <stevenknight>	it's controlled by setting MSVC_BATCH=True (or 1 or whatever)
 308 18:44:59 <garyo-home>	sounds great.  Does it generalize to other batch-buildable things?
 309 18:45:06 <stevenknight>	yes
 310 18:45:10 <GregoryNoel>	(I'm still reading, but I have to put down the burger to type.)
 311 18:45:18 <stevenknight>	okay, cool
 312 18:45:21 <Jason_>	if you can give me the detail of what to do i will give it a run on a few new i7 systems
 313 18:45:41 <stevenknight>	the Taskmaster issues is that this suggests the Taskmaster is handling the wrong granularity of input
 314 18:45:45 <stevenknight>	it's looking at individual Nodes
 315 18:45:55 <stevenknight>	and I'm beginning to think it should be looking at Executors
 316 18:46:13 <stevenknight>	GregoryNoel, just chew on that a little and let me know if it sounds crazy
 317 18:46:16 <stevenknight>	based on your TNG work
 318 18:46:37 <stevenknight>	you can do your own batch builder at the Action() level
 319 18:46:51 <stevenknight>	simplest case is Action('$FOOCOM', batch_key=True)
 320 18:47:02 <GregoryNoel>	That's what TNG does: the transverse graph is in (a wrapper around) the Executor.
 321 18:47:14 <stevenknight>	sweet, so I'm not entirely crazy, at least
 322 18:47:21 <stevenknight>	maybe this helps move us toward TNG
 323 18:47:52 <stevenknight>	batch_key=True will separate all calls to the given Builder using that action
 324 18:48:59 <stevenknight>	into grouped batches for the four-tuple (Action, env)
 325 18:49:05 <stevenknight>	excuse me, two-tuple
 326 18:49:22 <stevenknight>	batch_key can also be a function that takes (action, env, target, source)
 327 18:49:39 <stevenknight>	and returns the key to be used for separating batches
 328 18:50:37 <stevenknight>	so the MSVC_BATCH support uses the four-tuple (action, env, target.dir, source.dir) as the key
 329 18:50:56 <garyo-home>	That's very clever.
 330 18:51:05 <stevenknight>	with the upshot that you get batched compilation of all object files in a given directory
 331 18:51:26 <stevenknight>	because $CPPPPATH can affect your #includes based on the directory
 332 18:51:36 <garyo-home>	... and the same compiler options etc.
 333 18:51:49 <stevenknight>	right, out of the same env
 334 18:52:14 <garyo-home>	Is it a lot faster?
 335 18:52:31 <stevenknight>	for our config (Google Chrome) only about 10%-15%
 336 18:52:41 <stevenknight>	which is nothing to sneeze at, but still, we were hoping for more
 337 18:52:50 <garyo-home>	That's nothing to sneeze at (jinx)
 338 18:53:04 <stevenknight>	on the other hand, i haven't gone in and profiled and optimizd it yet
 339 18:53:17 <stevenknight>	i'll probably upload to a subversion branch so people can try it out early
 340 18:53:20 <garyo-home>	For us, we use -O3 with the Intel compiler so all the time goes into the link step, but I'm sure this will help us even in that case.
 341 18:53:21 <stevenknight>	and send out email+doc
 342 18:53:50 <stevenknight>	(i've been doing this work experimenting with mercurial as a front end private branch development)
 343 18:54:07 <stevenknight>	okay, enough of that
 344 18:54:24 <garyo-home>	what else?
 345 18:54:34 <stevenknight>	next:  Google contributors to SCons
 346 18:54:41 <Jason_>	so i have a question
 347 18:54:49 <stevenknight>	yes?
 348 18:55:03 <Jason_>	is there any plans to improve the depend checker for C/C++
 349 18:55:16 <Jason_>	there is a bit of an issue with BOOST headers for use
 350 18:55:21 <stevenknight>	i.e. make it like a real C preprocessor?
 351 18:55:34 <Jason_>	We have a work around... but it seem to cause more trouble than it solves
 352 18:55:39 <stevenknight>	there's an experimental module that will do that
 353 18:56:08 <Jason_>	well one that is smart enought to get the #include FOOBAR stuff
 354 18:56:16 <stevenknight>	yes
 355 18:56:27 <stevenknight>	take a look at (IIRC) Scanner/
 356 18:56:31 <Jason_>	how can we try it out
 357 18:56:59 <Jason_>	branch in SCons?
 358 18:57:01 <stevenknight>	there's some commented out code that uses a module to do enough "real" CPP stuff to handle boost
 359 18:57:05 <Jason_>	or in the main code
 360 18:57:09 <stevenknight>	no, it's checked in trunk
 361 18:57:33 <stevenknight>	just hasn't been productized by giving it the right user-configurable hook
 362 18:57:43 <stevenknight>	if you want to finish that piece and contribute it back, it'd be very welcome
 363 18:58:04 <stevenknight>	next:  google contributors
 364 18:58:30 <stevenknight>	i have more people here starting to look at SCons internals
 365 18:58:39 <stevenknight>	especially w.r.t. performance
 366 18:59:08 <stevenknight>	which is good
 367 18:59:23 <stevenknight>	but we're finding that a lot of the flexibility we prize in SCons
 368 18:59:53 <stevenknight>	is antithetical to the kind of caching we'd need to do to really speed up things
 369 19:00:08 <stevenknight>	i'm wrestling with how best to channel this interest and energy
 370 19:00:21 <garyo-home>	that's hard.
 371 19:00:34 <stevenknight>	in ways that help SCons overall as well as Google's own purposes
 372 19:00:50 <stevenknight>	a Google fork of SCons is unattractive
 373 19:00:58 <garyo-home>	Right, certainly.
 374 19:01:12 <garyo-home>	Some kind of "mode"?
 375 19:01:17 <stevenknight>	but taking some of this work back into main line runs serious risk of breaking backwards compatibility
 376 19:02:05 <garyo-home>	and also making some existing idioms just plain not work, possibly.
 377 19:02:12 <stevenknight>	right
 378 19:02:31 <garyo-home>	How invasive is it?
 379 19:02:57 <stevenknight>	well, we're in early stages
 380 19:03:07 <stevenknight>	so more hypthetical than not
 381 19:03:12 <garyo-home>	ok.
 382 19:03:27 <stevenknight>	one patch i have queued tries to avoid starving child worker threads
 383 19:03:39 <stevenknight>	(actually doesn't try, it does avoid it)
 384 19:03:57 <stevenknight>	by just feeding as many ready Tasks as possible into the queue
 385 19:04:00 <stevenknight>	that doesn't break compatibility
 386 19:04:20 <garyo-home>	That seems great.  In general, I think if it's really valuable, breaking simple back compat is ok (i.e. people have to do some simple recoding), but forcing larger changes is problematic.
 387 19:04:23 <stevenknight>	but it's in that pesky Taskmaster area where unintended affects abound
 388 19:04:25 <GregoryNoel>	Sounds like TNG
 389 19:05:00 <stevenknight>	mm, maybe I should send this patch to you to see what you think
 390 19:05:36 <stevenknight>	we were able to do some visualization (using Incredibuild) to see the starvation at work without the patch, and no starvation with it
 391 19:06:32 <GregoryNoel>	Sure, I can take a look.  You should also look at TNG.
 392 19:06:33 <stevenknight>	tradeoff is more CPU cycles in the master thread searching the whole DAG for work more frequently
 393 19:06:55 <GregoryNoel>	TNG avoids that.
 394 19:07:09 <stevenknight>	okay, let's swap code and see how close we are
 395 19:07:33 <stevenknight>	are there issues holding up TNG?
 396 19:08:02 <GregoryNoel>	No, not really.  Just lack of time.
 397 19:08:08 <stevenknight>	okay
 398 19:08:12 <stevenknight>	i'll send you the patch from here
 399 19:08:37 <stevenknight>	so now that I'm articulating this
 400 19:08:38 <GregoryNoel>	(here?)
 401 19:08:43 <stevenknight>	Google
 402 19:09:30 <stevenknight>	the meta issue is that this could bring a lot more impactive changes quickly into SCons
 403 19:09:42 <stevenknight>	with attendant possibility for destabilizing things
 404 19:10:04 <stevenknight>	in order to not hold up stuff, I'm going to have to spend (more) work time actually, oh, integrating patches
 405 19:10:07 <GregoryNoel>	(my fingers are still sticky, but I'm otherwise back.)
 406 19:10:08 <stevenknight>	in general, not just from Google
 407 19:10:34 <stevenknight>	and to do that I'm having to draft people here to cover some of the front-line work i've been trying to do on the Google Chrome build
 408 19:11:12 <stevenknight>	one other possibility I considered was to have a "google branch" in our SVN repository
 409 19:11:29 <stevenknight>	and pluck changes from that into trunk
 410 19:11:32 <GregoryNoel>	opposed: branches by features, not source
 411 19:11:49 <stevenknight>	yeah, that makes sense
 412 19:12:23 <stevenknight>	okay, so i just need to become johnny-on-the-spot with integration
 413 19:12:33 <stevenknight>	and maybe recruit additional committers / integrators
 414 19:12:40 <stevenknight>	from Google or elsewhere
 415 19:12:50 <garyo-home>	can you do that from Google?  Would be useful.
 416 19:13:05 <garyo-home>	We haven't had much luck from the users list to date.
 417 19:13:20 <stevenknight>	can I do...?  get more people from here working on SCons-the-project?
 418 19:13:57 <garyo-home>	Yes.
 419 19:14:18 <stevenknight>	there's certainly no institutional barrier
 420 19:14:57 <garyo-home>	Maybe get another googler or two on the dev list, tag along to a bug party...
 421 19:15:01 <stevenknight>	client-side projects here are getting behind SCons pretty heavily
 422 19:15:09 <GregoryNoel>	Doesn't Google have a rule that you can spend up to 20% of your time "doing good"?  Recruit in that space.
 423 19:15:43 <stevenknight>	that still ends up being driven by whether people have an itch to scratch
 424 19:15:51 <stevenknight>	that they want to spend their 20% on
 425 19:16:07 <garyo-home>	but SCons is *sooo* *coool*!
 426 19:16:07 <GregoryNoel>	Of course.  But fixing their build system to support themselves better is a strong itch.
 427 19:16:08 <stevenknight>	the potential itch here is speeding up builds for their projects
 428 19:16:31 <garyo-home>	That makes sense.
 429 19:16:53 <Jason_>	you need faster startup times?
 430 19:17:01 <stevenknight>	faster null build times
 431 19:17:14 <garyo-home>	steven: agreed, same here.
 432 19:17:18 <GregoryNoel>	ditto
 433 19:17:24 <garyo-home>	hence your caching ideas.
 434 19:17:28 <Jason_>	ya... big issue here.. I have a work around... but it a hack
 435 19:17:37 <stevenknight>	Jason_:  what's your workaround?
 436 19:17:48 <Jason_>	we have a DB file we make
 437 19:18:03 <Jason_>	we use this to figure out if we need to define soemthing to SCons
 438 19:18:13 <Jason_>	this speeds up builds a great deal
 439 19:18:26 <Jason_>	we woudl have NULL build of 8-10 minutes
 440 19:18:39 <stevenknight>	so you only conditionally load parts of the configuration based on this DB?
 441 19:18:47 <Jason_>	got it down to 13-63
 442 19:18:53 <Jason_>	depending on the target
 443 19:19:06 <Jason_>	13-63 secs
 444 19:19:22 <Jason_>	yes... but we have to have one good run
 445 19:19:33 <stevenknight>	right, to prep the DB with legitimate info
 446 19:19:35 <stevenknight>	that makes sense
 447 19:19:39 <Jason_>	else we don't know what is defined
 448 19:20:01 <stevenknight>	kind of related to this is making the DAG in SCons a real DAG
 449 19:20:08 <stevenknight>	(something I know Greg is very much in favor of)
 450 19:20:18 <stevenknight>	with separate tuples representing the edges
 451 19:20:19 <Jason_>	Same here
 452 19:20:35 <GregoryNoel>	Er, I thing you mean making the .sconsign a real DAG, but yes, I do.
 453 19:20:42 <Jason_>	then you can throw some well known task processing code at it
 454 19:20:53 <Jason_>	such as stuff used by a compiler
 455 19:20:59 <stevenknight>	right
 456 19:20:59 <garyo-home>	That scares me a little unless you explicitly allow runtime modification of the DAG
 457 19:21:16 <GregoryNoel>	white papers on request.
 458 19:21:27 <stevenknight>	and be able to infer what to build from known changed files
 459 19:21:43 <garyo-home>	... but maybe users have to explicitly identify when they're modifying the DAG at runtime to invalidate and/or rescan
 460 19:21:48 <stevenknight>	instead of always having to walk from targets down, searching for what might have changed
 461 19:21:57 <stevenknight>	GregoryNoel:  request, send white papers
 462 19:22:19 <stevenknight>	garyo-home:  you have a use case in mind?
 463 19:22:31 <garyo-home>	Dynamic source generators.
 464 19:22:42 <Jason_>	Agreeded
 465 19:22:46 <garyo-home>	Where you don't know all the generated source names up front.
 466 19:22:48 <Jason_>	we have that as well
 467 19:22:51 <stevenknight>	that should still be possible
 468 19:23:05 <garyo-home>	Good, I rely heavily on that.
 469 19:23:21 <stevenknight>	i think the nodes would still have .sources, .implicit, .explicit etc.
 470 19:23:23 <Jason_>	a simple example if a Doxygen builder
 471 19:23:31 <GregoryNoel>	In point of fact, I'd like to optimize the null case first, when the SConscripts haven't been changed, and work from there.
 472 19:23:33 <stevenknight>	but instead of pointing directly to ohter Nodes
 473 19:23:40 <Jason_>	it can't really know what it will build up front
 474 19:23:41 <stevenknight>	they'd be lists of DAG edges
 475 19:24:00 <stevenknight>	GregoryNoel:  agree in general
 476 19:24:03 <garyo-home>	Jason_: a very good use case there.
 477 19:24:18 <stevenknight>	but how do we know when the null case is optimized "enough" ?
 478 19:24:30 <garyo-home>	Steven: I have to think about that before I understand you.
 479 19:24:34 <GregoryNoel>	Zero time in parsing.
 480 19:24:50 <stevenknight>	wow, configurations are so different
 481 19:25:00 <stevenknight>	parsing is a really small part of our null build time
 482 19:25:32 <GregoryNoel>	I wonder...
 483 19:26:02 <stevenknight>	profiling shows we're heavily dominated by string (re-)expansion
 484 19:26:46 <GregoryNoel>	True, and working the bug list yesterday I found a case where a string was expanded *six* times to the same effect.
 485 19:26:35 <Jason_>	idea... you might want to look at the parts file
 486 19:26:59 <Jason_>	we sort of replace expantions.. to get around the time issue here
 487 19:27:07 <stevenknight>	Jason_;  aha
 488 19:27:27 <stevenknight>	I'll look at what you did as a source of ideas
 489 19:27:40 <stevenknight>	one of my big problems is I've spent so much time focusing on making corner cases work
 490 19:27:44 <Jason_>	part of this is me hacking a wrapper around the scanner object to force a few expansions
 491 19:27:59 <stevenknight>	that I'm prone to ignore effective solutions that really help the common case(s)
 492 19:28:03 <Jason_>	5-10% difference in speed
 493 19:28:32 <stevenknight>	we got a pretty good improvement just running big long CPPPATH lists through env.subst() when assigning them
 494 19:28:58 <Jason_>	the problem was i did not see it when you added it :-(
 495 19:29:13 <stevenknight>	?  not in SCons itself, in our configuration
 496 19:29:32 <Jason_>	oh... never was able to use them
 497 19:30:29 <stevenknight>	anyone else about ready to turn into a pumpkin too?
 498 19:30:36 <stevenknight>	I should head for home...
 499 19:30:39 <garyo-home>	yes, time for me to go.
 500 19:30:41 <GregoryNoel>	Long past time for me
 501 19:30:48 <stevenknight>	okay, many thanks guys
 502 19:30:52 <garyo-home>	Thanks a lot as usual, guys!
 503 19:30:54 <GregoryNoel>	two weeks?
 504 19:30:59 <garyo-home>	ok for me.
 505 19:31:00 <stevenknight>	hopefully we'll see some increased activity coming up
 506 19:31:06 <stevenknight>	two weeks is good for me
 507 19:31:07 <Jason_>	latter!
 508 19:31:12 <GregoryNoel>	Next meeting 17 Dec then?  1.2 should be out, as well as the first checkpoint toward 1.3.  Agreed?
 509 19:31:12 <GregoryNoel>	Next meeting 17 Dec then?  1.2 should be out, as well as the first checkpoint toward 1.3.  Agreed?
 510 19:31:12 <GregoryNoel>	Next meeting 17 Dec then?  1.2 should be out, as well as the first checkpoint toward 1.3.  Agreed?
 511 19:31:13 <stevenknight>	should have 1.2 out by then...
 512 19:31:27 <GregoryNoel>	oops...
 513 19:31:26 <stevenknight>	what I tell you three times is true...
 514 19:31:29 <garyo-home>	greg: yes, yes, yes.
 515 19:32:04 <stevenknight>	for the snark *was* a boojum, you see...
 516 19:32:08 <garyo-home>	:-)
 517 19:32:17 <GregoryNoel>	snicker-snack
 518 19:32:17 <garyo-home>	ok, g'night all.
 519 19:32:22 *	Jason_ has quit ("Leaving")
 520 19:32:25 <GregoryNoel>	cul
 521 19:32:29 <stevenknight>	later
 522 19:32:39 *	garyo-home has quit ("ChatZilla 0.9.84 [Firefox 3.0.4/2008102920]")
 523 19:47:01 *	stevenknight has quit ("Leaving")

BugParty/IrcLog2008-12-03 (last edited 2008-12-04 05:26:35 by ip68-7-77-81)