ci(actions): adjust test summary format #3588

kwonoj · 2023-02-01T22:57:31Z

closes WEB-541, test summary format adjustments.

vercel · 2023-02-01T22:57:34Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated
turbo-site	🔄 Building (Inspect)	Visit Preview		Feb 2, 2023 at 0:07AM (UTC)

9 Ignored Deployments

Name	Status	Updated
examples-basic-web	⬜️ Ignored (Inspect)	Feb 2, 2023 at 0:07AM (UTC)
examples-cra-web	⬜️ Ignored (Inspect)	Feb 2, 2023 at 0:07AM (UTC)
examples-designsystem-docs	⬜️ Ignored (Inspect)	Feb 2, 2023 at 0:07AM (UTC)
examples-kitchensink-blog	⬜️ Ignored (Inspect)	Feb 2, 2023 at 0:07AM (UTC)
examples-native-web	⬜️ Ignored (Inspect)	Feb 2, 2023 at 0:07AM (UTC)
examples-nonmonorepo	⬜️ Ignored (Inspect)	Feb 2, 2023 at 0:07AM (UTC)
examples-svelte-web	⬜️ Ignored (Inspect)	Feb 2, 2023 at 0:07AM (UTC)
examples-tailwind-web	⬜️ Ignored (Inspect)	Feb 2, 2023 at 0:07AM (UTC)
examples-vite-web	⬜️ Ignored (Inspect)	Feb 2, 2023 at 0:07AM (UTC)

github-actions · 2023-02-01T23:21:50Z

Failing next.js integration test suites

Commit: 434898d

Test summary

	main (`3d3f1a0` / v13.1.7-canary.2)	Current (`434898d` / v13.1.7-canary.2)	Diff (Failed)
Test suites	🔴 12 / 🟢 0 (Total: 12)	🔴 12 / 🟢 0 (Total: 12)	0️⃣
Test cases	🔴 20 / 🟢 16 (Total: 36)	🔴 20 / 🟢 16 (Total: 36)	0️⃣

Full test report

test/development/acceptance-app/component-stack.test.ts

Component Stack in error overlay > should show a component stack on hydration error

Expand output

● Component Stack in error overlay › should show a component stack on hydration error

page.waitForSelector: Timeout 30000ms exceeded.
=========================== logs ===========================
waiting for locator('[data-nextjs-toast]')
============================================================

  329 |     return this.chain(() => {
  330 |       return page
> 331 |         .waitForSelector(selector, { timeout, state: 'attached' })
      |          ^
  332 |         .then(async (el) => {
  333 |           // it seems selenium waits longer and tests rely on this behavior
  334 |           // so we wait for the load event fire before returning

  at lib/browsers/playwright.ts:331:10

test/e2e/handle-non-hoisted-swc-helpers/index.test.ts

handle-non-hoisted-swc-helpers > should work

Expand output

● handle-non-hoisted-swc-helpers › should work

expect(received).toContain(expected) // indexOf

Expected substring: "hello world"
Received string:    "failed to canonicalize embedded file path·
Caused by:
    No such file or directory (os error 2)"

  34 |   it('should work', async () => {
  35 |     const html = await renderViaHTTP(next.url, '/')
> 36 |     expect(html).toContain('hello world')
     |                  ^
  37 |   })
  38 | })
  39 |

  at Object.<anonymous> (e2e/handle-non-hoisted-swc-helpers/index.test.ts:36:18)

test/integration/bigint/test/index.test.js

bigint API route support > dev mode > should return 200
bigint API route support > dev mode > should return the BigInt result text

Expand output

● bigint API route support › dev mode › should return 200

expect(received).toEqual(expected) // deep equality

Expected: 200
Received: 500

  22 |       method: 'GET',
  23 |     })
> 24 |     expect(res.status).toEqual(200)
     |                        ^
  25 |   })
  26 |
  27 |   it('should return the BigInt result text', async () => {

  at Object.<anonymous> (integration/bigint/test/index.test.js:24:24)

● bigint API route support › dev mode › should return the BigInt result text

expect(received).toEqual(expected) // deep equality

Expected: "3"
Received: false

  29 |       method: 'GET',
  30 |     }).then((res) => res.ok && res.text())
> 31 |     expect(resText).toEqual('3')
     |                     ^
  32 |   })
  33 | }
  34 |

  at Object.<anonymous> (integration/bigint/test/index.test.js:31:21)

test/development/basic/legacy-decorators.test.ts

Legacy decorators SWC option > should compile with legacy decorators enabled

Expand output

● Legacy decorators SWC option › should compile with legacy decorators enabled

page.waitForSelector: Timeout 30000ms exceeded.
=========================== logs ===========================
waiting for locator('#count')
============================================================

  329 |     return this.chain(() => {
  330 |       return page
> 331 |         .waitForSelector(selector, { timeout, state: 'attached' })
      |          ^
  332 |         .then(async (el) => {
  333 |           // it seems selenium waits longer and tests rely on this behavior
  334 |           // so we wait for the load event fire before returning

  at lib/browsers/playwright.ts:331:10

test/e2e/app-dir/app-alias/app-alias.test.ts

dynamic-href > should error when using dynamic href.pathname in app dir
dynamic-href > should error when using dynamic href in app dir

Expand output

● dynamic-href › should error when using dynamic href.pathname in app dir

expect(received).toBeTrue()

Expected value to be true:
  true
Received:
  false

  14 |
  15 |         // Error should show up
> 16 |         expect(await hasRedbox(browser, true)).toBeTrue()
     |                                                ^
  17 |         expect(await getRedboxDescription(browser)).toMatchInlineSnapshot(
  18 |           `"Error: Dynamic href \`/object/[slug]\` found in <Link> while using the \`/app\` router, this is not supported. Read more: https://nextjs.org/docs/messages/app-dir-dynamic-href"`
  19 |         )

  at Object.<anonymous> (e2e/app-dir/dynamic-href/dynamic-href.test.ts:16:48)

● dynamic-href › should error when using dynamic href in app dir

expect(received).toBeTrue()

Expected value to be true:
  true
Received:
  false

  44 |
  45 |         // Error should show up
> 46 |         expect(await hasRedbox(browser, true)).toBeTrue()
     |                                                ^
  47 |         expect(await getRedboxDescription(browser)).toMatchInlineSnapshot(
  48 |           `"Error: Dynamic href \`/object/[slug]\` found in <Link> while using the \`/app\` router, this is not supported. Read more: https://nextjs.org/docs/messages/app-dir-dynamic-href"`
  49 |         )

  at Object.<anonymous> (e2e/app-dir/dynamic-href/dynamic-href.test.ts:46:48)

test/integration/404-page-custom-error/test/index.test.js

Default 404 Page with custom _error > dev mode > should respond to 404 correctly
Default 404 Page with custom _error > dev mode > should render error correctly
Default 404 Page with custom _error > dev mode > should render index page normal

Expand output

● Default 404 Page with custom _error › dev mode › should respond to 404 correctly

expect(received).toBe(expected) // Object.is equality

Expected: 404
Received: 500

  24 |   it('should respond to 404 correctly', async () => {
  25 |     const res = await fetchViaHTTP(appPort, '/404')
> 26 |     expect(res.status).toBe(404)
     |                        ^
  27 |     expect(await res.text()).toContain('This page could not be found')
  28 |   })
  29 |

  at Object.<anonymous> (integration/404-page-custom-error/test/index.test.js:26:24)

● Default 404 Page with custom _error › dev mode › should render error correctly

expect(received).toContain(expected) // indexOf

Expected substring: "oops"
Received string:    "failed to canonicalize embedded file path·
Caused by:
    No such file or directory (os error 2)"

  30 |   it('should render error correctly', async () => {
  31 |     const text = await renderViaHTTP(appPort, '/err')
> 32 |     expect(text).toContain(isDev ? 'oops' : 'Internal Server Error')
     |                  ^
  33 |   })
  34 |
  35 |   it('should render index page normal', async () => {

  at Object.<anonymous> (integration/404-page-custom-error/test/index.test.js:32:18)

● Default 404 Page with custom _error › dev mode › should render index page normal

expect(received).toContain(expected) // indexOf

Expected substring: "hello from index"
Received string:    "failed to canonicalize embedded file path·
Caused by:
    No such file or directory (os error 2)"

  35 |   it('should render index page normal', async () => {
  36 |     const html = await renderViaHTTP(appPort, '/')
> 37 |     expect(html).toContain('hello from index')
     |                  ^
  38 |   })
  39 |
  40 |   if (!isDev) {

  at Object.<anonymous> (integration/404-page-custom-error/test/index.test.js:37:18)

test/development/acceptance-app/component-stack.test.ts

Component Stack in error overlay > should show a component stack on hydration error

Expand output

● Component Stack in error overlay › should show a component stack on hydration error

page.waitForSelector: Timeout 30000ms exceeded.
=========================== logs ===========================
waiting for locator('[data-nextjs-toast]')
============================================================

  329 |     return this.chain(() => {
  330 |       return page
> 331 |         .waitForSelector(selector, { timeout, state: 'attached' })
      |          ^
  332 |         .then(async (el) => {
  333 |           // it seems selenium waits longer and tests rely on this behavior
  334 |           // so we wait for the load event fire before returning

  at lib/browsers/playwright.ts:331:10

test/e2e/handle-non-hoisted-swc-helpers/index.test.ts

handle-non-hoisted-swc-helpers > should work

Expand output

● handle-non-hoisted-swc-helpers › should work

expect(received).toContain(expected) // indexOf

Expected substring: "hello world"
Received string:    "failed to canonicalize embedded file path·
Caused by:
    No such file or directory (os error 2)"

  34 |   it('should work', async () => {
  35 |     const html = await renderViaHTTP(next.url, '/')
> 36 |     expect(html).toContain('hello world')
     |                  ^
  37 |   })
  38 | })
  39 |

  at Object.<anonymous> (e2e/handle-non-hoisted-swc-helpers/index.test.ts:36:18)

test/integration/bigint/test/index.test.js

bigint API route support > dev mode > should return 200
bigint API route support > dev mode > should return the BigInt result text

Expand output

● bigint API route support › dev mode › should return 200

expect(received).toEqual(expected) // deep equality

Expected: 200
Received: 500

  22 |       method: 'GET',
  23 |     })
> 24 |     expect(res.status).toEqual(200)
     |                        ^
  25 |   })
  26 |
  27 |   it('should return the BigInt result text', async () => {

  at Object.<anonymous> (integration/bigint/test/index.test.js:24:24)

● bigint API route support › dev mode › should return the BigInt result text

expect(received).toEqual(expected) // deep equality

Expected: "3"
Received: false

  29 |       method: 'GET',
  30 |     }).then((res) => res.ok && res.text())
> 31 |     expect(resText).toEqual('3')
     |                     ^
  32 |   })
  33 | }
  34 |

  at Object.<anonymous> (integration/bigint/test/index.test.js:31:21)

test/development/basic/legacy-decorators.test.ts

Legacy decorators SWC option > should compile with legacy decorators enabled

Expand output

● Legacy decorators SWC option › should compile with legacy decorators enabled

page.waitForSelector: Timeout 30000ms exceeded.
=========================== logs ===========================
waiting for locator('#count')
============================================================

  329 |     return this.chain(() => {
  330 |       return page
> 331 |         .waitForSelector(selector, { timeout, state: 'attached' })
      |          ^
  332 |         .then(async (el) => {
  333 |           // it seems selenium waits longer and tests rely on this behavior
  334 |           // so we wait for the load event fire before returning

  at lib/browsers/playwright.ts:331:10

test/e2e/app-dir/app-alias/app-alias.test.ts

dynamic-href > should error when using dynamic href.pathname in app dir
dynamic-href > should error when using dynamic href in app dir

Expand output

● dynamic-href › should error when using dynamic href.pathname in app dir

expect(received).toBeTrue()

Expected value to be true:
  true
Received:
  false

  14 |
  15 |         // Error should show up
> 16 |         expect(await hasRedbox(browser, true)).toBeTrue()
     |                                                ^
  17 |         expect(await getRedboxDescription(browser)).toMatchInlineSnapshot(
  18 |           `"Error: Dynamic href \`/object/[slug]\` found in <Link> while using the \`/app\` router, this is not supported. Read more: https://nextjs.org/docs/messages/app-dir-dynamic-href"`
  19 |         )

  at Object.<anonymous> (e2e/app-dir/dynamic-href/dynamic-href.test.ts:16:48)

● dynamic-href › should error when using dynamic href in app dir

expect(received).toBeTrue()

Expected value to be true:
  true
Received:
  false

  44 |
  45 |         // Error should show up
> 46 |         expect(await hasRedbox(browser, true)).toBeTrue()
     |                                                ^
  47 |         expect(await getRedboxDescription(browser)).toMatchInlineSnapshot(
  48 |           `"Error: Dynamic href \`/object/[slug]\` found in <Link> while using the \`/app\` router, this is not supported. Read more: https://nextjs.org/docs/messages/app-dir-dynamic-href"`
  49 |         )

  at Object.<anonymous> (e2e/app-dir/dynamic-href/dynamic-href.test.ts:46:48)

test/integration/404-page-custom-error/test/index.test.js

Default 404 Page with custom _error > dev mode > should respond to 404 correctly
Default 404 Page with custom _error > dev mode > should render error correctly
Default 404 Page with custom _error > dev mode > should render index page normal

Expand output

● Default 404 Page with custom _error › dev mode › should respond to 404 correctly

expect(received).toBe(expected) // Object.is equality

Expected: 404
Received: 500

  24 |   it('should respond to 404 correctly', async () => {
  25 |     const res = await fetchViaHTTP(appPort, '/404')
> 26 |     expect(res.status).toBe(404)
     |                        ^
  27 |     expect(await res.text()).toContain('This page could not be found')
  28 |   })
  29 |

  at Object.<anonymous> (integration/404-page-custom-error/test/index.test.js:26:24)

● Default 404 Page with custom _error › dev mode › should render error correctly

expect(received).toContain(expected) // indexOf

Expected substring: "oops"
Received string:    "failed to canonicalize embedded file path·
Caused by:
    No such file or directory (os error 2)"

  30 |   it('should render error correctly', async () => {
  31 |     const text = await renderViaHTTP(appPort, '/err')
> 32 |     expect(text).toContain(isDev ? 'oops' : 'Internal Server Error')
     |                  ^
  33 |   })
  34 |
  35 |   it('should render index page normal', async () => {

  at Object.<anonymous> (integration/404-page-custom-error/test/index.test.js:32:18)

● Default 404 Page with custom _error › dev mode › should render index page normal

expect(received).toContain(expected) // indexOf

Expected substring: "hello from index"
Received string:    "failed to canonicalize embedded file path·
Caused by:
    No such file or directory (os error 2)"

  35 |   it('should render index page normal', async () => {
  36 |     const html = await renderViaHTTP(appPort, '/')
> 37 |     expect(html).toContain('hello from index')
     |                  ^
  38 |   })
  39 |
  40 |   if (!isDev) {

  at Object.<anonymous> (integration/404-page-custom-error/test/index.test.js:37:18)

github-actions · 2023-02-02T01:15:33Z

Benchmark for 434898d

Test	Base	PR	%	Significant %
bench_startup/Turbopack RSC/1000 modules	2543.49ms ± 11.87ms	2630.74ms ± 14.24ms	+3.43%	+1.36%

Click to view full benchmark

Test	Base	PR	%	Significant %
bench_hmr_to_commit/Turbopack CSR/1000 modules	7696.91µs ± 57.89µs	7781.71µs ± 77.06µs	+1.10%
bench_hmr_to_commit/Turbopack RCC/1000 modules	8047.79µs ± 73.62µs	8133.79µs ± 117.66µs	+1.07%
bench_hmr_to_commit/Turbopack RSC/1000 modules	497.90ms ± 3.34ms	499.25ms ± 2.41ms	+0.27%
bench_hmr_to_commit/Turbopack SSR/1000 modules	7970.63µs ± 85.86µs	7957.08µs ± 115.59µs	-0.17%
bench_hmr_to_eval/Turbopack CSR/1000 modules	6785.50µs ± 45.26µs	6817.27µs ± 50.01µs	+0.47%
bench_hmr_to_eval/Turbopack RCC/1000 modules	7203.60µs ± 74.71µs	7265.95µs ± 97.55µs	+0.87%
bench_hmr_to_eval/Turbopack SSR/1000 modules	7155.83µs ± 94.26µs	7110.57µs ± 83.86µs	-0.63%
bench_hydration/Turbopack RCC/1000 modules	4132.59ms ± 18.27ms	4145.98ms ± 23.25ms	+0.32%
bench_hydration/Turbopack RSC/1000 modules	3558.50ms ± 16.29ms	3536.45ms ± 18.20ms	-0.62%
bench_hydration/Turbopack SSR/1000 modules	3409.59ms ± 21.21ms	3430.43ms ± 17.91ms	+0.61%
bench_startup/Turbopack CSR/1000 modules	2254.23ms ± 13.56ms	2262.26ms ± 14.80ms	+0.36%
bench_startup/Turbopack RCC/1000 modules	2794.27ms ± 9.59ms	2764.37ms ± 18.00ms	-1.07%
bench_startup/Turbopack RSC/1000 modules	2543.49ms ± 11.87ms	2630.74ms ± 14.24ms	+3.43%	+1.36%
bench_startup/Turbopack SSR/1000 modules	2307.50ms ± 11.70ms	2284.76ms ± 9.09ms	-0.99%

github-actions · 2023-02-02T03:21:14Z

⚠️ CI failed ⚠️

The following steps have failed in CI:

Turbopack Rust tests

See workflow summary for details

closes WEB-541, test summary format adjustments.

github-actions bot added the area: ci label Feb 1, 2023

vercel bot deployed to Preview – turbo-site February 1, 2023 22:58 View deployment

kwonoj force-pushed the format-passed-count branch 2 times, most recently from 21441c9 to 199dbab Compare February 1, 2023 23:47

ci(actions): adjust test summary format

eb50ad1

kwonoj force-pushed the format-passed-count branch from 199dbab to eb50ad1 Compare February 2, 2023 00:06

kwonoj changed the title ~~[DONOTMERGE] WIP~~ ci(actions): adjust test summary format Feb 2, 2023

kwonoj marked this pull request as ready for review February 2, 2023 00:07

kwonoj requested review from a team as code owners February 2, 2023 00:07

kwonoj requested review from chris-olszewski and arlyon February 2, 2023 00:07

sokra approved these changes Feb 2, 2023

View reviewed changes

kwonoj added the pr: automerge Kodiak will merge these automatically after checks pass label Feb 2, 2023

arlyon approved these changes Feb 2, 2023

View reviewed changes

sokra merged commit 4e9a7c7 into main Feb 3, 2023

sokra deleted the format-passed-count branch February 3, 2023 10:23

mehulkar pushed a commit that referenced this pull request Feb 3, 2023

ci(actions): adjust test summary format (#3588)

91bef69

closes WEB-541, test summary format adjustments.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

ci(actions): adjust test summary format #3588

ci(actions): adjust test summary format #3588

Uh oh!

kwonoj commented Feb 1, 2023 •

edited

Loading

Uh oh!

vercel bot commented Feb 1, 2023 •

edited

Loading

Uh oh!

github-actions bot commented Feb 1, 2023 •

edited

Loading

Uh oh!

github-actions bot commented Feb 2, 2023

Uh oh!

github-actions bot commented Feb 2, 2023

Uh oh!

Uh oh!

ci(actions): adjust test summary format #3588

ci(actions): adjust test summary format #3588

Uh oh!

Conversation

kwonoj commented Feb 1, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vercel bot commented Feb 1, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 1, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Failing next.js integration test suites

Test summary

Uh oh!

github-actions bot commented Feb 2, 2023

Benchmark for 434898d

Uh oh!

github-actions bot commented Feb 2, 2023

⚠️ CI failed ⚠️

Uh oh!

Uh oh!

kwonoj commented Feb 1, 2023 •

edited

Loading

vercel bot commented Feb 1, 2023 •

edited

Loading

github-actions bot commented Feb 1, 2023 •

edited

Loading