Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

deposit in prologue and refund in epilogue to prevent undergasing attack for randomness txns #12695

Merged
merged 22 commits into from Apr 17, 2024

Conversation

zjma
Copy link
Contributor

@zjma zjma commented Mar 27, 2024

Description

  • New steps in a user transaction execution.
    • Before prologue, determine if the txn requires deposit, if so how much.
    • At the end of prologue, if deposit is required, burn the calculated amount from the gas payer's balance.
    • At the beginning of epilogue, if deposit was required, mint and refund the same amount back to the gas payer.
  • For randomness transactions, define the required deposit as min(A, B), where:
    • A = (max_execution_gas + max_io_gas) * price_per_gas_unit + max_storage_fee, and
    • B = maximum_number_of_gas_units * price_per_gas_unit.
    • This is a prevention of the undergasing attack.

Some reference numbers on mainnet.

max_execution_gas=920
max_io_gas=1_000
typical_octa_per_gas=100
max_storage_fee=200_000_000
max_number_of_gas_units=2_000_000

Therefore, for a txn that directly or indirectly call randomness API, the required deposit is 2_000_000 octas, i.e., 2APT.

Type of Change

  • New feature

Which Components or Systems Does This Change Impact?

  • Move/Aptos Virtual Machine

How Has This Been Tested?

UTs

Key Areas to Review

function execute_user_transaction_impl.

Checklist

  • I have read and followed the CONTRIBUTING doc
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I identified and added all stakeholders and component owners affected by this change as reviewers
  • I tested both happy and unhappy path of the functionality
  • I have made corresponding changes to the documentation

Copy link

trunk-io bot commented Mar 27, 2024

⏱️ 26h 7m total CI duration on this PR
Job Cumulative Duration Recent Runs
rust-unit-tests 6h 27m 🟩🟩🟩 (+15 more)
rust-move-unit-coverage 4h 44m 🟩🟩🟩 (+14 more)
execution-performance / single-node-performance 2h 58m 🟩🟩🟩🟩🟩 (+3 more)
windows-build 2h 40m 🟩🟩🟩🟩🟩 (+4 more)
rust-smoke-tests 2h 35m 🟩🟩 (+1 more)
rust-lints 1h 52m 🟥🟩🟩🟩 (+15 more)
run-tests-main-branch 1h 21m 🟩🟩🟩🟩🟩 (+15 more)
check 58m 🟩🟩🟩🟩🟩 (+11 more)
check-dynamic-deps 46m 🟩🟩🟩🟩🟩 (+16 more)
general-lints 40m 🟩🟩🟩🟩🟩 (+15 more)
rust-move-tests 27m 🟩🟩🟩🟩🟩 (+15 more)
semgrep/ci 10m 🟩🟩🟩🟩🟩 (+16 more)
file_change_determinator 5m 🟩🟩🟩🟩🟩 (+15 more)
permission-check 5m 🟩🟩🟩🟩🟩 (+14 more)
file_change_determinator 4m 🟩🟩🟩🟩🟩 (+16 more)
file_change_determinator 4m 🟩🟩🟩🟩🟩 (+14 more)
rust-images / rust-all 2m 🟥🟥🟥🟥🟥 (+1 more)
permission-check 2m 🟩🟩🟩🟩🟩 (+15 more)
file_change_determinator 2m 🟩🟩🟩🟩🟩 (+2 more)
execution-performance / file_change_determinator 2m 🟩🟩🟩🟩🟩 (+2 more)
permission-check 1m 🟩🟩🟩🟩🟩 (+14 more)
permission-check 1m 🟩🟩🟩🟩🟩 (+15 more)
permission-check 30s 🟩🟩🟩🟩🟩 (+2 more)
determine-docker-build-metadata 20s 🟩🟩🟩🟩🟩 (+2 more)

🚨 3 jobs on the last run were significantly faster/slower than expected

Job Duration vs 7d avg Delta
rust-smoke-tests 38m 32m +21%
rust-lints 5m 7m -24%
windows-build 12m 20m -40%

settingsfeedbackdocs ⋅ learn more about trunk.io

Copy link

codecov bot commented Mar 27, 2024

Codecov Report

Attention: Patch coverage is 0% with 128 lines in your changes are missing coverage. Please review.

Project coverage is 62.6%. Comparing base (7647a94) to head (72e83b7).
Report is 23 commits behind head on main.

❗ Current head 72e83b7 differs from pull request most recent head c62b5c1. Consider uploading reports for the commit c62b5c1 to get more accurate results

Files Patch % Lines
aptos-move/aptos-vm/src/aptos_vm.rs 0.0% 61 Missing ⚠️
aptos-move/aptos-vm/src/transaction_validation.rs 0.0% 60 Missing ⚠️
aptos-move/aptos-vm/src/transaction_metadata.rs 0.0% 4 Missing ⚠️
..._ext/session/user_transaction_sessions/prologue.rs 0.0% 2 Missing ⚠️
aptos-move/aptos-vm/src/errors.rs 0.0% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff            @@
##             main   #12695     +/-   ##
=========================================
- Coverage    62.6%    62.6%   -0.1%     
=========================================
  Files         823      820      -3     
  Lines      184343   184003    -340     
=========================================
- Hits       115499   115251    -248     
+ Misses      68844    68752     -92     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@zjma zjma marked this pull request as ready for review March 27, 2024 17:17
@zjma zjma requested review from msmouse and danielxiangzl and removed request for davidiw, junkil-park, vgao1996, wrwg and movekevin March 27, 2024 17:17
aptos-move/aptos-vm/src/aptos_vm.rs Outdated Show resolved Hide resolved
aptos-move/aptos-vm/src/transaction_validation.rs Outdated Show resolved Hide resolved
@@ -294,11 +423,15 @@ module aptos_framework::transaction_validation {
transaction_fee_amount
};

if (amount_to_burn > storage_fee_refunded) {
let burn_amount = amount_to_burn - storage_fee_refunded;
let refund = storage_fee_refunded;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

assertion on line 407 seems not proper any more

also, line 417 is probably broken

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why? i thought the only change to the epilogue is to mint back?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should consider the burnt amount when judging if the sender has enough fund to cover the gas?

Copy link
Contributor Author

@zjma zjma Mar 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so randomness refund should happen earilier, somewhere before L407?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you should do the math with the refund accounted for and still either mint or refund once.

@zjma zjma force-pushed the zjma/protection-against-gas-attack branch from 8f87bfb to 539b358 Compare March 28, 2024 18:27
@davidiw
Copy link
Contributor

davidiw commented Mar 28, 2024

So now that I had more time to think about this, why aren't we just making this the status quo for all transactions?

@@ -36,10 +36,14 @@ pub static APTOS_TRANSACTION_VALIDATION: Lazy<TransactionValidation> =
module_addr: CORE_CODE_ADDRESS,
module_name: Identifier::new("transaction_validation").unwrap(),
fee_payer_prologue_name: Identifier::new("fee_payer_script_prologue").unwrap(),
fee_payer_prologue_v2_name: Identifier::new("fee_payer_script_prologue_v2").unwrap(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can upgrade private functions iirc, so this code can be avoided

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we update the function arg list inline, then the rust side needs an additional feature flag to know when to include the required_deposit arg, which i thought is worse than defining a func v2 + using v2 only when required_deposit.is_some (so no additional feature flag is needed)?

cc @zekun000

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought we need a flag anyway, we can't use required_deposit to decide if calling v1 or v2 because the framework may not be updated yet

| TransactionPayload::ModuleBundle(_)
| TransactionPayload::Multisig(_) => false,
TransactionPayload::EntryFunction(entry_func) => {
has_randomness_attribute(resolver, session, entry_func).unwrap_or(false)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should propagate the error, which can happen if module loading failed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(i'm turning this func into get_required_deposit)
can this function focus on required deposit calculation and let a later step capture the module loading failure?

@@ -2407,12 +2445,15 @@ impl VMValidator for AptosVM {
let mut session = self.new_session(&resolver, SessionId::prologue_meta(&txn_data));

// Increment the counter for transactions verified.
let has_randomness_annotation =
check_randomness_annotation(&mut session, &resolver, txn.payload());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When you do let txn_data = TransactionMetadata::new(&txn); few lines above you can probably also add a field there for has_randomness_annotation? Then no need to pass it around. Also calling more generally makes more sense if we want to switch to this flow at some point anyway?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we now have required_deposit in TransactionMetadata.

aptos-move/aptos-vm/src/aptos_vm.rs Outdated Show resolved Hide resolved
aptos-move/aptos-vm/src/aptos_vm.rs Outdated Show resolved Hide resolved
aptos-move/aptos-vm/src/aptos_vm.rs Show resolved Hide resolved
@@ -136,9 +162,36 @@ module aptos_framework::transaction_validation {
);
let balance = coin::balance<AptosCoin>(gas_payer);
assert!(balance >= max_transaction_fee, error::invalid_argument(PROLOGUE_ECANT_PAY_GAS_DEPOSIT));

if (option::is_some(&required_deposit)) {
transaction_fee::burn_fee(gas_payer, option::extract(&mut required_deposit));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need a new error code if the max_transaction_fee != required_deposit, otherwise the abort will be Unexpected prologue Move abort https://github.com/aptos-labs/aptos-core/blob/main/aptos-move/aptos-vm/src/errors.rs#L64

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how does the comparison between max_transaction_fee and required_deposit matter?
did you mean a new error code for when there's not enough balance to deposit?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

new error code added

@@ -104,7 +112,15 @@ pub(crate) fn run_script_prologue(
MoveValue::U64(txn_expiration_timestamp_secs),
MoveValue::U8(chain_id.id()),
];
(&APTOS_TRANSACTION_VALIDATION.fee_payer_prologue_name, args)
if required_deposit.is_some() {
args.push(required_deposit.as_move_value());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this works since the binary and framework are not updated at the same time, we need to handle the case that the framework is not updated

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed by gating randomness deposit calculation with randomness feature flags (and in the future. feature X deposit calculation with X feature flags)
?

@@ -189,40 +210,65 @@ fn run_epilogue(
fee_statement: FeeStatement,
txn_data: &TransactionMetadata,
features: &Features,
required_deposit: Option<u64>,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: maybe we can merge this into TransactionMetadata so we don't need to pass the extra param all over the place

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we now have required_deposit in TransactionMetadata.

let internal_gas_per_gas = u64::from(txn_gas_params.scaling_factor());
let max_execution_gas = Gas::from(
u64::from(txn_gas_params.max_execution_gas).div_ceil(internal_gas_per_gas),
);
Copy link
Contributor Author

@zjma zjma Mar 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vgao1996 is there a better way to convert InternalGas to Gas?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe just

                    let max_execution_gas: Gas = txn_gas_params
                        .max_execution_gas
                        .to_unit_round_up_with_params(txn_gas_params);

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

aptos-move/aptos-vm/src/aptos_vm.rs Show resolved Hide resolved
aptos-move/aptos-vm/src/aptos_vm.rs Outdated Show resolved Hide resolved
let internal_gas_per_gas = u64::from(txn_gas_params.scaling_factor());
let max_execution_gas = Gas::from(
u64::from(txn_gas_params.max_execution_gas).div_ceil(internal_gas_per_gas),
);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe just

                    let max_execution_gas: Gas = txn_gas_params
                        .max_execution_gas
                        .to_unit_round_up_with_params(txn_gas_params);

@zjma zjma added CICD:run-e2e-tests when this label is present github actions will run all land-blocking e2e tests from the PR CICD:run-framework-upgrade-test labels Mar 31, 2024
txn.payload(),
)
} else {
return VMValidatorResult::error(StatusCode::UNKNOWN_VALIDATION_STATUS);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this error code fine?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zjma zjma changed the title pre-pay fees for randomness transactions deposit in prologue and refund in epilogue for randomness (and future features) Apr 1, 2024
@zjma zjma changed the title deposit in prologue and refund in epilogue for randomness (and future features) deposit in prologue and refund in epilogue to prevent undergasing attack for randomness txns Apr 1, 2024
@zjma zjma force-pushed the zjma/protection-against-gas-attack branch from a69d8a5 to cf9ebc5 Compare April 17, 2024 01:57

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

✅ Forge suite compat success on aptos-node-v1.10.1 ==> c62b5c1c7b6a8a7aea6326c4567c8859f733df0b

Compatibility test results for aptos-node-v1.10.1 ==> c62b5c1c7b6a8a7aea6326c4567c8859f733df0b (PR)
1. Check liveness of validators at old version: aptos-node-v1.10.1
compatibility::simple-validator-upgrade::liveness-check : committed: 6616 txn/s, latency: 5018 ms, (p50: 4800 ms, p90: 8600 ms, p99: 9900 ms), latency samples: 231580
2. Upgrading first Validator to new version: c62b5c1c7b6a8a7aea6326c4567c8859f733df0b
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1716 txn/s, latency: 16894 ms, (p50: 19100 ms, p90: 22200 ms, p99: 22500 ms), latency samples: 89280
3. Upgrading rest of first batch to new version: c62b5c1c7b6a8a7aea6326c4567c8859f733df0b
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1484 txn/s, latency: 19214 ms, (p50: 22000 ms, p90: 26900 ms, p99: 28100 ms), latency samples: 78700
4. upgrading second batch to new version: c62b5c1c7b6a8a7aea6326c4567c8859f733df0b
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 3624 txn/s, latency: 8805 ms, (p50: 9600 ms, p90: 12600 ms, p99: 12800 ms), latency samples: 144980
5. check swarm health
Compatibility test for aptos-node-v1.10.1 ==> c62b5c1c7b6a8a7aea6326c4567c8859f733df0b passed
Test Ok

Copy link
Contributor

✅ Forge suite realistic_env_max_load success on c62b5c1c7b6a8a7aea6326c4567c8859f733df0b

two traffics test: inner traffic : committed: 7423 txn/s, latency: 5285 ms, (p50: 5100 ms, p90: 6000 ms, p99: 11000 ms), latency samples: 3229020
two traffics test : committed: 100 txn/s, latency: 2073 ms, (p50: 1900 ms, p90: 2200 ms, p99: 6600 ms), latency samples: 1760
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.205, avg: 0.201", "QsPosToProposal: max: 0.273, avg: 0.249", "ConsensusProposalToOrdered: max: 0.471, avg: 0.443", "ConsensusOrderedToCommit: max: 0.380, avg: 0.363", "ConsensusProposalToCommit: max: 0.816, avg: 0.806"]
Max round gap was 1 [limit 4] at version 921949. Max no progress secs was 4.582133 [limit 15] at version 921949.
Test Ok

@zjma zjma merged commit f585507 into main Apr 17, 2024
48 of 51 checks passed
@zjma zjma deleted the zjma/protection-against-gas-attack branch April 17, 2024 12:28
Copy link
Contributor

❌ Forge suite framework_upgrade failure on aptos-node-v1.10.1 ==> c62b5c1c7b6a8a7aea6326c4567c8859f733df0b

Compatibility test results for aptos-node-v1.10.1 ==> c62b5c1c7b6a8a7aea6326c4567c8859f733df0b (PR)
Upgrade the nodes to version: c62b5c1c7b6a8a7aea6326c4567c8859f733df0b
Test Failed: API error: Unknown error Transaction committed on chain, but failed execution: LOOKUP_FAILED

Stack backtrace:
   0: anyhow::error::<impl core::convert::From<E> for anyhow::Error>::from
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.79/src/error.rs:565:25
   1: <core::result::Result<T,F> as core::ops::try_trait::FromResidual<core::result::Result<core::convert::Infallible,E>>>::from_residual
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/result.rs:1963:27
   2: aptos_release_builder::validate::NetworkConfig::execute_proposal::{{closure}}
             at ./aptos-move/aptos-release-builder/src/validate.rs:364:9
   3: aptos_release_builder::validate::NetworkConfig::submit_and_execute_multi_step_proposal::{{closure}}
             at ./aptos-move/aptos-release-builder/src/validate.rs:124:64
   4: aptos_release_builder::validate::execute_release::{{closure}}
             at ./aptos-move/aptos-release-builder/src/validate.rs:432:22
   5: aptos_release_builder::validate::validate_config_and_generate_release::{{closure}}
             at ./aptos-move/aptos-release-builder/src/validate.rs:493:6
   6: aptos_release_builder::validate::validate_config::{{closure}}
             at ./aptos-move/aptos-release-builder/src/validate.rs:479:80
   7: tokio::runtime::park::CachedParkThread::block_on::{{closure}}
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/park.rs:282:63
   8: tokio::runtime::coop::with_budget
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/coop.rs:107:5
   9: tokio::runtime::coop::budget
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/coop.rs:73:5
  10: tokio::runtime::park::CachedParkThread::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/park.rs:282:31
  11: tokio::runtime::context::blocking::BlockingRegionGuard::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/context/blocking.rs:66:9
  12: tokio::runtime::scheduler::multi_thread::MultiThread::block_on::{{closure}}
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/scheduler/multi_thread/mod.rs:87:13
  13: tokio::runtime::context::runtime::enter_runtime
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/context/runtime.rs:65:16
  14: tokio::runtime::scheduler::multi_thread::MultiThread::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/scheduler/multi_thread/mod.rs:86:9
  15: tokio::runtime::runtime::Runtime::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/runtime.rs:350:50
  16: <aptos_testcases::framework_upgrade::FrameworkUpgrade as aptos_forge::interface::network::NetworkTest>::run
             at ./testsuite/testcases/src/framework_upgrade.rs:97:9
  17: aptos_forge::runner::Forge<F>::run::{{closure}}
             at ./testsuite/forge/src/runner.rs:598:42
  18: aptos_forge::runner::run_test
             at ./testsuite/forge/src/runner.rs:666:11
  19: aptos_forge::runner::Forge<F>::run
             at ./testsuite/forge/src/runner.rs:598:30
  20: forge::run_forge
             at ./testsuite/forge-cli/src/main.rs:427:11
  21: forge::main
             at ./testsuite/forge-cli/src/main.rs:353:21
  22: core::ops::function::FnOnce::call_once
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ops/function.rs:250:5
  23: std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/sys_common/backtrace.rs:154:18
  24: std::rt::lang_start::{{closure}}
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:167:18
  25: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ops/function.rs:284:13
  26: std::panicking::try::do_call
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:552:40
  27: std::panicking::try
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
  28: std::panic::catch_unwind
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
  29: std::rt::lang_start_internal::{{closure}}
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:48
  30: std::panicking::try::do_call
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:552:40
  31: std::panicking::try
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
  32: std::panic::catch_unwind
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
  33: std::rt::lang_start_internal
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:20
  34: std::rt::lang_start
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:166:17
  35: __libc_start_main
  36: _start
Trailing Log Lines:
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
  32: std::panic::catch_unwind
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
  33: std::rt::lang_start_internal
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:20
  34: std::rt::lang_start
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:166:17
  35: __libc_start_main
  36: _start


Swarm logs can be found here: See fgi output for more information.
{"level":"INFO","source":{"package":"aptos_forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:292"},"thread_name":"main","hostname":"forge-framework-upgrade-pr-12695-1713355831-aptos-node-v1-10-1","timestamp":"2024-04-17T12:58:54.217905Z","message":"Deleting namespace forge-framework-upgrade-pr-12695: Some(NamespaceStatus { conditions: None, phase: Some(\"Terminating\") })"}
{"level":"INFO","source":{"package":"aptos_forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:400"},"thread_name":"main","hostname":"forge-framework-upgrade-pr-12695-1713355831-aptos-node-v1-10-1","timestamp":"2024-04-17T12:58:54.217931Z","message":"aptos-node resources for Forge removed in namespace: forge-framework-upgrade-pr-12695"}

failures:
Failed to run tests:
Tests Failed
    framework_upgrade::framework-upgrade

test result: FAILED. 0 passed; 1 failed; 0 filtered out

Error: Tests Failed

Stack backtrace:
   0: anyhow::error::<impl anyhow::Error>::msg
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.79/src/error.rs:83:36
   1: aptos_forge::runner::Forge<F>::run
             at ./testsuite/forge/src/runner.rs:618:13
   2: forge::run_forge
             at ./testsuite/forge-cli/src/main.rs:427:11
   3: forge::main
             at ./testsuite/forge-cli/src/main.rs:353:21
   4: core::ops::function::FnOnce::call_once
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ops/function.rs:250:5
   5: std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/sys_common/backtrace.rs:154:18
   6: std::rt::lang_start::{{closure}}
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:167:18
   7: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ops/function.rs:284:13
   8: std::panicking::try::do_call
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:552:40
   9: std::panicking::try
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
  10: std::panic::catch_unwind
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
  11: std::rt::lang_start_internal::{{closure}}
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:48
  12: std::panicking::try::do_call
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:552:40
  13: std::panicking::try
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
  14: std::panic::catch_unwind
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
  15: std::rt::lang_start_internal
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:20
  16: std::rt::lang_start
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:166:17
  17: __libc_start_main
  18: _start
Debugging output:
NAME                       READY   STATUS    RESTARTS   AGE
aptos-node-0-validator-0   1/1     Running   0          44m
aptos-node-1-validator-0   1/1     Running   0          43m
aptos-node-2-validator-0   1/1     Running   0          43m
aptos-node-3-validator-0   1/1     Running   0          42m

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CICD:run-e2e-tests when this label is present github actions will run all land-blocking e2e tests from the PR CICD:run-framework-upgrade-test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

9 participants