1035551 - Add in_tree_config support for Marionette

Status: RESOLVED FIXED

(Remote Protocol :: Marionette, defect)

Description

[Gregory Szorc [:gps]]

The mozharness Marionette configs don't currently define in_tree_config. We need this to help roll out bug 940954.

Comment 1

[Gregory Szorc [:gps]]

Attachment: Support in-tree configs for Marionette

We'll be adding a new argument to the Marionette CLI driver in bug
940954. We'll need to have mozharness pass this argument to get tests to
work properly. To facilitate this, we're adding an in-tree Marionette
config that will add this argument.

Reviewer Note: I'm not sure if I changed too many or too few of the
Marionette config files to now include an in-tree config. Also, I'm
pretty sure mozharness fails if the in-tree config file isn't present.
So I guess I need to add an empty file to all the trees?

As much as I wanted to use the existing mac_config.py, linux_config.py,
etc config files, marionette seems to share config files between
platforms. I'm not sure if I should be doing os.platform() foo inside
the config file or what. That's what review is for!

Comment 2

[Jonathan Griffin (:jgriffin)]

Comment on attachment 8452776 [details] [diff] [review]
Support in-tree configs for Marionette

Review of attachment 8452776 [details] [diff] [review]:
-----------------------------------------------------------------

lgtm

Comment 4

[Gregory Szorc [:gps]]

Chris: If you want to steal this from me, go right ahead. I'll be side-tracked for the next few days dealing with Firefox foo.

Comment 5

[Andrew Halberstadt [:ahal]]

Chris, you'll still need to pull the command line arguments out of marionette.py and into the config. Also, all the other harness' in-tree configs live under testing/config/mozharness, can we put this file alongside those?

Comment 6

[Chris Manchester (limited bugmail, email directly)]

Attachment: Add a config in tree for marionette tests.

This is looking good on ash.

Comment 7

[Chris Manchester (limited bugmail, email directly)]

Attachment: Add an in-tree config for marionette tests.

This is the config file to be used.

Comment 8

[Chris Manchester (limited bugmail, email directly)]

Attachment: Add support for in tree configs in mozharness in marionette.py script.

These are the modifications to the mozharness script to read the in-tree config.

Comment 9

[Andrew Halberstadt [:ahal]]

Comment on attachment 8455411 [details] [diff] [review]
Add an in-tree config for marionette tests.

Review of attachment 8455411 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good!

Comment 10

[Andrew Halberstadt [:ahal]]

Comment on attachment 8455413 [details] [diff] [review]
Add support for in tree configs in mozharness in marionette.py script.

Review of attachment 8455413 [details] [diff] [review]:
-----------------------------------------------------------------

Nice, thanks! So I see you are going the 'land on every tree' route. I agree this is a good idea since this time all you are doing is adding a file that should get automatically copied over since the ground work was already laid out by bug 981030. If you ask one of the sheriffs nicely, they might be able to help you uplift to all of the older branches since they do uplifts there once in awhile anyway. Failing that, I can help you do it.

One thing to note, people who haven't re-based their Mn try runs will hit a failure. Luckily from last time an appropriate error message should already get printed if this happens:
http://mxr.mozilla.org/build/source/mozharness/mozharness/mozilla/testing/testbase.py#327

We'll just want to update the referenced bug to note that Mn was recently changed.

Comment 11

[Chris Manchester (limited bugmail, email directly)]

Attachment: Add an in-tree config file for marionette for all branches.

Here's a version of the config suitable for all branches (doesn't add the "--log-raw" option).

r=ahal

Comment 12

[Chris Manchester (limited bugmail, email directly)]

As we did in bug 981030, we need to land the new config on every tree so that the mozharness change doesn't cause bustage. ahal recommended I alert a sheriff to see about how to do this.

The attachment in comment 11 is intended to land on all the trees.

Ryan, what do you think?

Comment 13

[Chris Manchester (limited bugmail, email directly)]

Config for central landed:

https://hg.mozilla.org/integration/mozilla-inbound/rev/3053c94b3215

Comment 14

[Chris Manchester (limited bugmail, email directly)]

Attachment: Add support for in tree configs in mozharness in marionette.py script.

Updated error message to point to this bug.

r=ahal

Comment 15

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/3053c94b3215

Comment 16

[Chris Manchester (limited bugmail, email directly)]

This has been on central about a day, let's get the ball rolling. 

Ryan, could you land the patch from comment 11 on all trees when you have an opportunity?

Comment 17

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/dab59c1307c4
https://hg.mozilla.org/releases/mozilla-release/rev/650879b8b29d
https://hg.mozilla.org/releases/mozilla-esr31/rev/02284260fbb4
https://hg.mozilla.org/releases/mozilla-b2g30_v1_4/rev/074223c547af
https://hg.mozilla.org/releases/mozilla-b2g28_v1_3/rev/88fc67bb7d2b
https://hg.mozilla.org/releases/mozilla-esr24/rev/785034ef8ed5

Comment 18

[Chris Manchester (limited bugmail, email directly)]

(In reply to Ryan VanderMeulen [:RyanVM UTC-4] from comment #17)
> https://hg.mozilla.org/releases/mozilla-aurora/rev/dab59c1307c4
> https://hg.mozilla.org/releases/mozilla-release/rev/650879b8b29d
> https://hg.mozilla.org/releases/mozilla-esr31/rev/02284260fbb4
> https://hg.mozilla.org/releases/mozilla-b2g30_v1_4/rev/074223c547af
> https://hg.mozilla.org/releases/mozilla-b2g28_v1_3/rev/88fc67bb7d2b
> https://hg.mozilla.org/releases/mozilla-esr24/rev/785034ef8ed5

This is the config with the new option for logging, so will cause bustage once we land the mozharness changes. We need the config from comment 11 without this option, not the one checked into central: https://bug1035551.bugzilla.mozilla.org/attachment.cgi?id=8455507

I know this is a hassle, sorry for the confusion here!

Comment 19

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/4c74eca8112f
https://hg.mozilla.org/releases/mozilla-release/rev/a1a032b080d3
https://hg.mozilla.org/releases/mozilla-esr31/rev/8879005f2bd6
https://hg.mozilla.org/releases/mozilla-b2g30_v1_4/rev/68a2a43dff5b
https://hg.mozilla.org/releases/mozilla-b2g28_v1_3/rev/a7dd18f55543
https://hg.mozilla.org/releases/mozilla-esr24/rev/31c2006018e1

Comment 20

[Chris Manchester (limited bugmail, email directly)]

Mozharness changes:

https://hg.mozilla.org/build/mozharness/rev/589cda99ef76

Comment 21

[Andrew Halberstadt [:ahal]]

Here is a green cypress run with the above patch included:
https://tbpl.mozilla.org/?tree=Cypress&jobname=marionette&rev=0dc711216018

Comment 22

[Chris Manchester (limited bugmail, email directly)]

Looks like this hit production here:

http://hg.mozilla.org/build/mozharness/rev/1c4d5ded5f51

Comment 23

[Chris Manchester (limited bugmail, email directly)]

Re-triggers from this morning on the older trees look ok, except for release where there isn't anything recent enough on release I can re-trigger with all the changes picked up.